Commit Graph

249 Commits (fbc66e3ab35743cc847a21223c67d88bb463cd9c)

Author SHA1 Message Date
bashonly c6387abc1a
[cleanup] Misc ()
Closes , Closes , Closes 
Authored by: bashonly, Codenade, pzhlkj6612, seproDev, coletdjnz, grqz, Grub4K

Co-authored-by: Codenade <amadeus.dorian04@gmail.com>
Co-authored-by: Mozi <29089388+pzhlkj6612@users.noreply.github.com>
Co-authored-by: sepro <4618135+seproDev@users.noreply.github.com>
Co-authored-by: coletdjnz <coletdjnz@protonmail.com>
Co-authored-by: N/Ame <173015200+grqz@users.noreply.github.com>
Co-authored-by: Simon Sawicki <contact@grub4k.xyz>
bashonly 6aaf96a3d6
[cleanup] Misc ()
Closes 
Authored by: bashonly, seproDev, jucor, c-basalt

Co-authored-by: sepro <4618135+seproDev@users.noreply.github.com>
Co-authored-by: Julien Cornebise <julien@cornebise.com>
Co-authored-by: c-basalt <117849907+c-basalt@users.noreply.github.com>
bashonly ca8885edd9 [fd/hls] Apply `extra_param_to_key_url` from info dict
Authored by: bashonly
sepro add96eb9f8
[cleanup] Add more ruff rules ()
Authored by: seproDev

Reviewed-by: bashonly <88596187+bashonly@users.noreply.github.com>
Reviewed-by: Simon Sawicki <contact@grub4k.xyz>
Simon Sawicki 9590cc6b47
Add new option `--progress-delta` ()
Authored by: Grub4K
bashonly 9c42b7eef5
[fd/ffmpeg] Accept output args from info dict ()
Authored by: bashonly
Peter Hosey 2d91b98456
[fd/http] Reset resume length to handle `FileNotFoundError` ()
Closes 
Authored by: boredzo
sepro f24e44e8cb
[webvtt] Don't parse single fragment files ()
Partially addresses 
Authored by: seproDev
coletdjnz ccfd70f4c2
[rh:websockets] Migrate websockets to networking framework ()
* Adds a basic WebSocket framework
* Introduces new minimum `websockets` version of 12.0
* Deprecates `WebSocketsWrapper`

Fixes https://github.com/yt-dlp/yt-dlp/issues/8439

Authored by: coletdjnz
bashonly 2622c804d1
[fd/dash] Force native downloader for `--live-from-start` ()
Closes 
Authored by: bashonly
CrendKing 21b25281c5
[fd/aria2c] Remove duplicate `--file-allocation=none` ()
Authored by: CrendKing
Simon Sawicki 1c51c520f7
[fd/fragment] Improve progress calculation ()
This uses the download speed from all threads and also adds smoothing to speed and eta

Authored by: Grub4K
pukkandan 62b5c94cad
[cleanup] Misc fixes
Closes 
bashonly e57eb98222
[fd/external] Fix ffmpeg input from stdin ()
Bugfix for 1ceb657bdd

Authored by: bashonly
bashonly 42ded0a429
[fd/external] Fixes to cookie handling
- Fix bug in `axel` Cookie header arg
- Pass cookies to `curl` as strings
- Write session cookies for `aria2c` and `wget`

Closes 
Authored by: bashonly
bashonly 6c5211cebe
[core] Fix HTTP headers and cookie handling
- Remove `Cookie` header from `http_headers` immediately after loading into cookiejar
- Restore compat for `--load-info-json` cookies
- Add more tests
- Fix improper passing of Cookie header by `MailRu` extractor

Closes 
Authored by: bashonly, pukkandan
coletdjnz 3d2623a898
[compat, networking] Deprecate old functions ()
Authored by: coletdjnz, pukkandan
coletdjnz 227bf1a33b
[networking] Rewrite architecture ()
New networking interface consists of a `RequestDirector` that directs
each `Request` to appropriate `RequestHandler` and returns the
`Response` or raises `RequestError`. The handlers define adapters to
transform its internal Request/Response/Errors to our interfaces.

User-facing changes:
- Fix issues with per request proxies on redirects for urllib
- Support for `ALL_PROXY` environment variable for proxy setting
- Support for `socks5h` proxy
   - Closes https://github.com/yt-dlp/yt-dlp/issues/6325, https://github.com/ytdl-org/youtube-dl/issues/22618, https://github.com/ytdl-org/youtube-dl/pull/28093
- Raise error when using `https` proxy instead of silently converting it to `http`

Authored by: coletdjnz
Simon Sawicki 3121512228
[core] Change how `Cookie` headers are handled
Cookies are now saved and loaded under `cookies` key in the info dict
instead of `http_headers.Cookie`. Cookies passed in headers are
auto-scoped to the input URLs with a warning.

Ref: https://github.com/yt-dlp/yt-dlp/security/advisories/GHSA-v8mc-9377-rwjj

Authored by: Grub4K
bashonly 1ceb657bdd
[fd/external] Scope cookies
- ffmpeg: Calculate cookies from cookiejar and pass with `-cookies` arg instead of `-headers`
- aria2c, curl, wget: Write cookiejar to file and use external FD built-in cookiejar support
- httpie: Calculate cookies from cookiejar instead of `http_headers`
- axel: Calculate cookies from cookiejar and disable http redirection if cookies are passed
    - May break redirects, but axel simply don't have proper cookie support

Ref: https://github.com/yt-dlp/yt-dlp/security/advisories/GHSA-v8mc-9377-rwjj

Authored by: bashonly, coletdjnz
pukkandan bc344cd456
[core] Allow extractors to mark formats as potentially DRM ()
This is useful for HLS where detecting whether the format is
actually DRM requires the child manifest to be downloaded.

Makes the error message when using `--test` inconsistent,
but doesn't really matter.
pukkandan 337734d4a8
[cleanup] Misc
pukkandan 662ef1e910
[downloader/http] Avoid infinite loop when no data is received
Closes 
pukkandan 8a8af356e3
[downloader/aria2c] Add `--no-conf`
Closes 
pukkandan ad54c9130e
[cleanup] Misc
Closes , Closes , Closes , Closes , Closes 
Authored by: mikf, freezboltz, pukkandan
pukkandan 424f3bf033
[downloader/fragment] Do not sleep between fragments
Closes 
Lesmiscore f8f9250fe2
[extractor/niconico:live] Add extractor ()
Authored by: Lesmiscore
Simon Sawicki edbe5b589d
Bugfixes for 4823ec9f46
Hotfix for fragmented downloads

Authored by: bashonly
Simon Sawicki 8417f26b8a
[core] Implement `--color` flag ()
Authored by: Grub4K
pukkandan 4823ec9f46
Update to ytdl-commit-d1c6c5
[YouTube] [core] Improve platform debug log, based on yt-dlp
d1c6c5c4d6

Except:
    * 6ed34338285f722d0da312ce0af3a15a077a3e2a [jsinterp] Add short-cut evaluation for common expression
        * There was no performance improvement when tested with https://github.com/ytdl-org/youtube-dl/issues/30641
    * e8de54bce50f6f77a4d7e8e80675f7003d5bf630 [core] Handle `/../` sequences in HTTP URLs
        * We plan to implement this differently
pukkandan 46f1370e9a
[devscripts/cli_to_api] Add script
coletdjnz 955c89584b
[core] Deprecate internal `Youtubedl-no-compression` header ()
Authored by: coletdjnz
pukkandan 66aeaac9aa
[downloader/curl] Fix progress reporting
Bug in 8c53322cda
Closes 
pukkandan 5b28cef72d
[cleanup] Misc
pukkandan f34804b2f9
[extractor/youtube] Fix 5038f6d713
* [fragment] Fix `request_data`
* [youtube] Don't use POST for now. It may be easier to break in future

Authored by: bashonly, coletdjnz
pukkandan 65f6e80780
[dependencies] Simplify `Cryptodome`
Closes , closes , closes 
Felix Yan 65e5c021e7
[utils] Don't use Content-length with encoding ()
Authored by: felixonmars
Closes , 
Simon Sawicki 6839ae1f6d
[utils] `traverse_obj`: Fix more bugs
and cleanup uses of `default=[]`

Continued from b1bde57bef
bashonly 7e68567e50
[downloader/hls] Allow extractors to provide AES key ()
and related cleanup

Authored by: bashonly, Grub4K

Co-authored-by: Simon Sawicki <contact@grub4k.xyz>
pukkandan f6a765ceb5
[dependencies] Standardize `Cryptodome` imports
pukkandan ad68b16a1e
[downloader/aria2c] Disable native progress
Closes , closes , Re-opens 
Lesmiscore 8c53322cda
[downloader/aria2c] Native progress for aria2c via RPC ()
Authored by: Lesmiscore, pukkandan

Closes 
bashonly 3cf50fa8e9
[downloader/ffmpeg] Fix headers for video+audio formats ()
Authored by: bashonly, Grub4K
pukkandan 71df9b7fd5
[cleanup] Misc
bashonly 3b021eacef
[extractor/generic] Add `fragment_query` extractor arg for DASH and HLS ()
* `fragment_query`: passthrough any query in generic mpd/m3u8 manifest URLs to their fragments
* Add support for `extra_param_to_segment_url` to DASH downloader
Authored by: bashonly, pukkandan
pukkandan 64c464a144
[utils] Move `FileDownloader.parse_bytes` into utils
pukkandan d5d1df8afd
[cleanup Misc
Closes 
pukkandan 814bba3933
[downloader/fragment] HLS download can continue without first fragment
Closes 
pukkandan a057779d5e
[cleanup] Minor fixes
Closes , Closes 
std-move af7a5eef2f
[downloader/aria2c] Fix filename containing leading whitespace ()
Similar to eb55bad5a0, but for fragmented downloads
Authored by: std-move