Commit Graph

554 Commits (e17e2beea64ba063fef6ff36fdf3d602fcbe2cec)

Author SHA1 Message Date
bashonly 75dc8e673b
[networking] Fix `--legacy-server-connect` ()
Bugfix for 227bf1a33b

Authored by: bashonly
bashonly 6c5211cebe
[core] Fix HTTP headers and cookie handling
- Remove `Cookie` header from `http_headers` immediately after loading into cookiejar
- Restore compat for `--load-info-json` cookies
- Add more tests
- Fix improper passing of Cookie header by `MailRu` extractor

Closes 
Authored by: bashonly, pukkandan
Aaruni Kaushik 2b029ca0a9
[cleanup] Add color to `download-archive` message ()
Authored by: aaruni96, Grub4K, pukkandan
Closes 
coletdjnz 3d2623a898
[compat, networking] Deprecate old functions ()
Authored by: coletdjnz, pukkandan
coletdjnz 227bf1a33b
[networking] Rewrite architecture ()
New networking interface consists of a `RequestDirector` that directs
each `Request` to appropriate `RequestHandler` and returns the
`Response` or raises `RequestError`. The handlers define adapters to
transform its internal Request/Response/Errors to our interfaces.

User-facing changes:
- Fix issues with per request proxies on redirects for urllib
- Support for `ALL_PROXY` environment variable for proxy setting
- Support for `socks5h` proxy
   - Closes https://github.com/yt-dlp/yt-dlp/issues/6325, https://github.com/ytdl-org/youtube-dl/issues/22618, https://github.com/ytdl-org/youtube-dl/pull/28093
- Raise error when using `https` proxy instead of silently converting it to `http`

Authored by: coletdjnz
pukkandan c365dba843
[networking] Add module ()
No actual changes - code is only moved around
Simon Sawicki 3121512228
[core] Change how `Cookie` headers are handled
Cookies are now saved and loaded under `cookies` key in the info dict
instead of `http_headers.Cookie`. Cookies passed in headers are
auto-scoped to the input URLs with a warning.

Ref: https://github.com/yt-dlp/yt-dlp/security/advisories/GHSA-v8mc-9377-rwjj

Authored by: Grub4K
pukkandan bc344cd456
[core] Allow extractors to mark formats as potentially DRM ()
This is useful for HLS where detecting whether the format is
actually DRM requires the child manifest to be downloaded.

Makes the error message when using `--test` inconsistent,
but doesn't really matter.
pukkandan 906c0bdcd8
[formats] Fix best fallback for storyboards
Partial fix for 
pukkandan 337734d4a8
[cleanup] Misc
pukkandan 47bcd43724
[outtmpl] Pad `playlist_index` etc even when with internal formatting
Closes 
pukkandan b4e0d75848
Improve `--download-sections`
* Support negative time-ranges
* Add `*from-url` to obey time-ranges in URL

Closes 
pukkandan 0dff8e4d1e
Indicate `filesize` approximated from `tbr` better
pukkandan 1619ab3e67
Bugfix for ebe1b4e34f
pukkandan 84078a8b38
[core] Fix `filepath` being copied to underlying format dict
Closes 
pukkandan ad54c9130e
[cleanup] Misc
Closes , Closes , Closes , Closes , Closes 
Authored by: mikf, freezboltz, pukkandan
Nicolai Dagestad db3ad8a676
Add option `--netrc-cmd` ()
Authored by: NDagestad, pukkandan
Closes 
pukkandan ebe1b4e34f
[outtmpl] Fix some minor bugs
Closes 
pukkandan 93b39cdbd9
Add `--compat-option playlist-match-filter`
Closes 
Ivan Skodje 372a0f3b9d
Auto-select default format in `-f-` ()
Authored by: ivanskodje, pukkandan
Closes 
coletdjnz 3f66b6fe50
[core] Workaround erroneous urllib Windows proxy parsing ()
Convert proxies extracted from windows registry to http for older Python versions.
See: https://github.com/python/cpython/issues/86793

Authored by: coletdjnz
coletdjnz b87e01c123
[cookies] Move `YoutubeDLCookieJar` to cookies module ()
Authored by: coletdjnz
Simon Sawicki 8417f26b8a
[core] Implement `--color` flag ()
Authored by: Grub4K
pukkandan 46f1370e9a
[devscripts/cli_to_api] Add script
coletdjnz 955c89584b
[core] Deprecate internal `Youtubedl-no-compression` header ()
Authored by: coletdjnz
coletdjnz 69bec6730e
[cleanup, utils] Split into submodules ()
Closes https://github.com/yt-dlp/yt-dlp/pull/2173

Authored by: pukkandan, coletdjnz
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
pukkandan 6f2287cb18
[cleanup] Misc
Closes , closes 
pukkandan c8bc203fbf
[docs] Misc improvements
Closes , closes , closes , closes , closes , closes , closes 
pukkandan 17ba4343cf
Fix f005a35aa7
Printing inside `finally` causes the order of logging to change
when there is an error, which is undesirable. So this is reverted.

The issue of `--print` being blocked by pre-processors was an
unintentional side-effect of changing the operation orders in
170605840e, and this is also partially
reverted.
pukkandan f005a35aa7
Ensure pre-processor errors do not block `--print`
Closes 
pukkandan b5f61b69d4
Fix bug in 170605840e
and related refactor
pukkandan 170605840e
Populate `filename` and `urls` fields at all stages of `--print`
Closes https://github.com/yt-dlp/yt-dlp/issues/6920
pukkandan ec9311c41b
[outtmpl] Support `str.format` syntax inside replacements
Closes 
pukkandan 78fde6e339
[outtmpl] Allow `\n` in replacements and default.
Fixes: https://github.com/yt-dlp/yt-dlp/issues/6808#issuecomment-1510055357
Fixes: https://github.com/yt-dlp/yt-dlp/issues/6808#issuecomment-1510363645
pukkandan 9874e82b5a
Do not translate newlines in `--print-to-file`
Fixes https://github.com/yt-dlp/yt-dlp/issues/6808#issuecomment-1509361107
pukkandan c3f624ef0a
Relaxed validation for numeric format filters
Continued from f96bff99cb

Closes 
pukkandan 26010b5cec
[postprocessor/FixupDuplicateMoov] Fix bug in triggering
pukkandan ab1de9cb1e
Support loading info.json with a list at it's root
pukkandan 392389b7df
[cleanup] Misc
pukkandan fe2ce85aff
Add option `--break-match-filters`
* Deprecates `--break-on-reject`

Closes 
Simon Sawicki 29cb20bd56
[build] Automated builds and nightly releases ()
Closes 
Authored by: Grub4K, bashonly

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
pukkandan 5b28cef72d
[cleanup] Misc
pukkandan a538772969
[cleanup] Misc
Closes 
Siddhartha Sahu 376aa24b15
Improve default subtitle language selection ()
Authored by: sdht0
pukkandan 9ebac35577
Bugfix for 39f32f1715
when `--ignore-no-formats-error`
pukkandan c154302c58
Bugfix for 39f32f1715
pukkandan 39f32f1715
Sanitize formats before sorting
Closes 
bashonly 7e68567e50
[downloader/hls] Allow extractors to provide AES key ()
and related cleanup

Authored by: bashonly, Grub4K

Co-authored-by: Simon Sawicki <contact@grub4k.xyz>
pukkandan 7aefd19afe
Make `title` completely non-fatal
Ref: https://github.com/yt-dlp/yt-dlp/pull/6158#discussion_r1096984349
pukkandan 59d7de0da5
Fix `--concat-playlist`
Closes 
pukkandan f079514957
[utils] `windows_enable_vt_mode`: Better error handling
Closes 
pukkandan 08e29b9f1f
[cleanup] Misc
Closes , closes 
Matthew 8300774c4a
Add `--enable-file-urls` ()
Closes https://github.com/yt-dlp/yt-dlp/issues/3675

Authored by: coletdjnz
Matthew e756f45ba0
Improve handling for overriding extractors with plugins ()
* Extractors replaced with plugin extractors now show in debug output
* Better testcase handling
* Added documentation
Authored by: coletdjnz, pukkandan
pukkandan 193fb150b7
Fix bug in 119e40ef64
pukkandan 88fb942577
Add message when there are no subtitles/thumbnails
Closes 
Matthew 8e40b9d1ec
Improve plugin architecture ()
to make plugins easier to develop and use:
* Plugins are now loaded as namespace packages.
* Plugins can be loaded in any distribution of yt-dlp (binary, pip, source, etc.).
* Plugin packages can be installed and managed via pip, or dropped into any of the documented locations.
* Users do not need to edit any code files to install plugins.
* Backwards-compatible with previous plugin architecture.

As a side-effect, yt-dlp will now search in a few more locations for config files.

Closes https://github.com/yt-dlp/yt-dlp/issues/1389

Authored by: flashdagger, coletdjnz, pukkandan, Grub4K
Co-authored-by: Marcel <flashdagger@googlemail.com>
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
Co-authored-by: Simon Sawicki <accounts@grub4k.xyz>
pukkandan ec54bd43f3
Fix bug in writing playlist info-json
Closes 
pukkandan 119e40ef64
Add pre-processor stage `video`
Related: , 
Anant Murmu efa944f4bc
[cleanup] Use `random.choices` ()
Authored by: freezboltz
pukkandan 8791e78ccc
Fix `original_url` in playlists
pukkandan 71df9b7fd5
[cleanup] Misc
pukkandan 784320c98c
Implement universal format sorting
Closes 
pukkandan 105bfd90f5
Add new field `aspect_ratio`
Closes 
pukkandan 6368e2e639
[cleanup] Misc
Closes 
pukkandan d7b460d0e5
Make early reject of `--match-filter` stricter
Closes 
pukkandan bc5c2f8a2c
Fix bugs in `PlaylistEntries`
bashonly e4221b700f
Fix `--list` options not implying `-s` in some cases ()
Authored by: bashonly, Grub4K
pukkandan 497074f044
Write API params in debug head
pukkandan d5d1df8afd
[cleanup Misc
Closes 
cruel-efficiency 2576d53a31
Fix end time of clips ()
Closes 
Authored by: cruel-efficiency
pukkandan 9b9dad119a
[outtmpl] Ensure ASCII in json and add option for Unicode
Closes 
pukkandan 94dc8604dd
Do more processing in `--flat-playlist`
pukkandan aebb4f4ba7
Fix for formats=None
Fixes: https://github.com/yt-dlp/yt-dlp/pull/4965#issuecomment-1267682512
pukkandan a057779d5e
[cleanup] Minor fixes
Closes , Closes 
sam 177662e0f2
[extractor/MicrosoftEmbed] Add extractor ()
Closes 
Authored by: DoubleCouponDay
Simon Sawicki dfea94f8f6
[extractor/crunchyroll:beta] Improve handling of hardsubs ()
Closes 
Authored by: Grub4K
pukkandan 0500ee3d81
Don't download entire video when no matching `--download-sections`
pukkandan 0bd5a039ea
Playlists maynot always have webpage_url
Lesmiscore fc2ba496fd
Allow open ranges for time ranges ()
Authored by: Lesmiscore
pukkandan 9c935fbc72
Fix bug in ae1035646a
Closes 
pukkandan deae7c1711
[cleanup] Misc
pukkandan 941e881e1f
Fix bug in ae1035646a
Closes 
pukkandan ae1035646a
Allow a `set` to be passed as `download_archive`
pukkandan 17ffed1842
[docs] Improvements
* Move detailed installation instructions to https://github.com/yt-dlp/yt-dlp/wiki/Installation
* Link to wiki where applicable
* Fix some mistakes. Closes , Closes , Closes 
* Improve some error messages
pukkandan 48c8424bd9
Fix bug in 07a1250e0e
pukkandan 07a1250e0e
[outtmpl] Curly braces to filter keys
pukkandan d2c8aadf79
[cleanup] Misc
Closes , Closes , Closes 
Authored by: pukkandan, MrRawes, DavidH-2022
pukkandan 1ac7f46184
Update to ytdl-commit-ed5c44e7
[compat] Replace deficient ChainMap class in Py3.3 and earlier
ed5c44e7b7
bashonly 9bd13fe5bb
[cookies] Support firefox container in `--cookies-from-browser` ()
Authored by: bashonly
pukkandan da4db748fa
[utils] Add `deprecation_warning`
See https://github.com/yt-dlp/yt-dlp/pull/2173#issuecomment-1097021515
pukkandan b5e7a2e69d
Add version to infojson
pukkandan fd404bec7e
Fix `--break-per-url --max-downloads`
pukkandan fe7866d0ed
Add option `--use-extractors`
Deprecates `--force-generic-extractor`

Closes , Closes 

Related: , 
pukkandan 5314b52192
[utils] Add orderedSet_from_options
pukkandan a831c2ea90
[cleanup] Misc
pukkandan c200096c03
Fix bug in --download-archive
Closes 
pukkandan 1e4fca9a87
[cleanup] Misc
Lesmiscore 62b58c0936
[docs] Consistent use of `e.g.` ()
Authored by: Lesmiscore
pukkandan f0ad6f8c51
Remove filtered entries from `-J`
Closes