Commit Graph

231 Commits (3121512228487c9c690d3d39bfd2579addf96e07)

Author SHA1 Message Date
Simon Sawicki 3121512228
[core] Change how `Cookie` headers are handled
Cookies are now saved and loaded under `cookies` key in the info dict
instead of `http_headers.Cookie`. Cookies passed in headers are
auto-scoped to the input URLs with a warning.

Ref: https://github.com/yt-dlp/yt-dlp/security/advisories/GHSA-v8mc-9377-rwjj

Authored by: Grub4K
bashonly 1ceb657bdd
[fd/external] Scope cookies
- ffmpeg: Calculate cookies from cookiejar and pass with `-cookies` arg instead of `-headers`
- aria2c, curl, wget: Write cookiejar to file and use external FD built-in cookiejar support
- httpie: Calculate cookies from cookiejar instead of `http_headers`
- axel: Calculate cookies from cookiejar and disable http redirection if cookies are passed
    - May break redirects, but axel simply don't have proper cookie support

Ref: https://github.com/yt-dlp/yt-dlp/security/advisories/GHSA-v8mc-9377-rwjj

Authored by: bashonly, coletdjnz
pukkandan bc344cd456
[core] Allow extractors to mark formats as potentially DRM ()
This is useful for HLS where detecting whether the format is
actually DRM requires the child manifest to be downloaded.

Makes the error message when using `--test` inconsistent,
but doesn't really matter.
pukkandan 337734d4a8
[cleanup] Misc
pukkandan 662ef1e910
[downloader/http] Avoid infinite loop when no data is received
Closes 
pukkandan 8a8af356e3
[downloader/aria2c] Add `--no-conf`
Closes 
pukkandan ad54c9130e
[cleanup] Misc
Closes , Closes , Closes , Closes , Closes 
Authored by: mikf, freezboltz, pukkandan
pukkandan 424f3bf033
[downloader/fragment] Do not sleep between fragments
Closes 
Lesmiscore f8f9250fe2
[extractor/niconico:live] Add extractor ()
Authored by: Lesmiscore
Simon Sawicki edbe5b589d
Bugfixes for 4823ec9f46
Hotfix for fragmented downloads

Authored by: bashonly
Simon Sawicki 8417f26b8a
[core] Implement `--color` flag ()
Authored by: Grub4K
pukkandan 4823ec9f46
Update to ytdl-commit-d1c6c5
[YouTube] [core] Improve platform debug log, based on yt-dlp
d1c6c5c4d6

Except:
    * 6ed34338285f722d0da312ce0af3a15a077a3e2a [jsinterp] Add short-cut evaluation for common expression
        * There was no performance improvement when tested with https://github.com/ytdl-org/youtube-dl/issues/30641
    * e8de54bce50f6f77a4d7e8e80675f7003d5bf630 [core] Handle `/../` sequences in HTTP URLs
        * We plan to implement this differently
pukkandan 46f1370e9a
[devscripts/cli_to_api] Add script
coletdjnz 955c89584b
[core] Deprecate internal `Youtubedl-no-compression` header ()
Authored by: coletdjnz
pukkandan 66aeaac9aa
[downloader/curl] Fix progress reporting
Bug in 8c53322cda
Closes 
pukkandan 5b28cef72d
[cleanup] Misc
pukkandan f34804b2f9
[extractor/youtube] Fix 5038f6d713
* [fragment] Fix `request_data`
* [youtube] Don't use POST for now. It may be easier to break in future

Authored by: bashonly, coletdjnz
pukkandan 65f6e80780
[dependencies] Simplify `Cryptodome`
Closes , closes , closes 
Felix Yan 65e5c021e7
[utils] Don't use Content-length with encoding ()
Authored by: felixonmars
Closes , 
Simon Sawicki 6839ae1f6d
[utils] `traverse_obj`: Fix more bugs
and cleanup uses of `default=[]`

Continued from b1bde57bef
bashonly 7e68567e50
[downloader/hls] Allow extractors to provide AES key ()
and related cleanup

Authored by: bashonly, Grub4K

Co-authored-by: Simon Sawicki <contact@grub4k.xyz>
pukkandan f6a765ceb5
[dependencies] Standardize `Cryptodome` imports
pukkandan ad68b16a1e
[downloader/aria2c] Disable native progress
Closes , closes , Re-opens 
Lesmiscore 8c53322cda
[downloader/aria2c] Native progress for aria2c via RPC ()
Authored by: Lesmiscore, pukkandan

Closes 
bashonly 3cf50fa8e9
[downloader/ffmpeg] Fix headers for video+audio formats ()
Authored by: bashonly, Grub4K
pukkandan 71df9b7fd5
[cleanup] Misc
bashonly 3b021eacef
[extractor/generic] Add `fragment_query` extractor arg for DASH and HLS ()
* `fragment_query`: passthrough any query in generic mpd/m3u8 manifest URLs to their fragments
* Add support for `extra_param_to_segment_url` to DASH downloader
Authored by: bashonly, pukkandan
pukkandan 64c464a144
[utils] Move `FileDownloader.parse_bytes` into utils
pukkandan d5d1df8afd
[cleanup Misc
Closes 
pukkandan 814bba3933
[downloader/fragment] HLS download can continue without first fragment
Closes 
pukkandan a057779d5e
[cleanup] Minor fixes
Closes , Closes 
std-move af7a5eef2f
[downloader/aria2c] Fix filename containing leading whitespace ()
Similar to eb55bad5a0, but for fragmented downloads
Authored by: std-move
nixxo 81b6102d20
[downloader/ism] Support ec-3 codec ()
Closes 
Authored by: nixxo
pukkandan deae7c1711
[cleanup] Misc
pukkandan da4db748fa
[utils] Add `deprecation_warning`
See https://github.com/yt-dlp/yt-dlp/pull/2173#issuecomment-1097021515
pukkandan e1eabd7beb
[downloader/external] Smarter detection of executable
Closes 
Lesmiscore 62b58c0936
[docs] Consistent use of `e.g.` ()
Authored by: Lesmiscore
pukkandan be5c1ae862
Standardize retry mechanism ()
* [utils] Create `RetryManager`
* Migrate all retries to use the manager
* [extractor] Add wrapper methods for convenience
* Standardize console messages for retries
* Add `--retry-sleep` for extractors
pukkandan 3df4f81dfe
[downloader] Add average speed to final progress line
Fixes: https://github.com/ytdl-org/youtube-dl/issues/31122
pukkandan b4daacb4ec
[mhtml, cleanup] Use imghdr
pukkandan 6929b41a21
Remove Python 3.6 support
Closes 
pukkandan f5ea47488a
[cleanup] Minor fixes
pukkandan 6d645b5577
[http] Ensure the file handle is always closed
Closes 
pukkandan ae61d108dd
[cleanup] Misc cleanup
pukkandan bbae437723
[hls] Warn user when trying to download live HLS
We do not automatically switch to ffmpeg because the detection is not 100% accurate
pukkandan 14f25df2b6
[compat] Remove deprecated functions from core code
pukkandan 54007a45f1
[cleanup] Consistent style for file heads
pukkandan ac66811112
[compat] Remove more functions
Removing any more will require changes to a large number of extractors
pukkandan 998a3cae0c
[cleanup] Misc fixes
pukkandan 95032f302c
[f4m] Bugfix