Commit Graph

163 Commits (80c360d7aad7ccda48ccd13be62dbb6fc5a6f128)

Author SHA1 Message Date
pukkandan e37d0efbd9
Fix bug where `original_url` was not propagated when `_type`=`url`
Matt Broadway 982ee69a74
Add option `--cookies-from-browser` to load cookies from a browser ()
* also adds `--no-cookies-from-browser`

Original PR: https://github.com/ytdl-org/youtube-dl/pull/29201
Authored by: mbway
pukkandan ae30b84072
Add field `live_status`
pukkandan cc9d1493c6
bugfix for 50fed816dd
pukkandan b35496d825
Add `only_once` param for `write_debug`
pukkandan c84aeac6b5
Add `only_once` param for `report_warning`
Related: https://github.com/yt-dlp/yt-dlp/pull/488#discussion_r667527297
pukkandan 50fed816dd
Errors in playlist extraction should obey `--ignore-errors`
Related: https://github.com/yt-dlp/yt-dlp/issues/535#issuecomment-883277272, https://github.com/yt-dlp/yt-dlp/issues/518#issuecomment-881794754
pukkandan 3f698246b2
Rename `NOTE` in `-F` to `MORE INFO`
since it's often confused to be the same as `format_note`
pukkandan cca80fe611 [youtube] Extract even more thumbnails and reduce testing
* Also fix bug where `_test_url` was being ignored

Ref: https://stackoverflow.com/a/20542029
Related: 
pukkandan 0ba692acc8
[youtube] Extract more thumbnails
* The thumbnail URLs are hard-coded and their actual existence is tested lazily
* Added option `--no-check-formats` to not test them

Closes , Related: , , https://github.com/ytdl-org/youtube-dl/issues/29049
pukkandan 3b297919e0
Revert "Merge webm formats into mkv if thumbnails are to be embedded ()"
This reverts commit 4d971a16b8 by @damianoamatruda
Closes 

This was wrongly checking for `write_thumbnail`
pukkandan b5ac45b197
Fix selectors `all`, `mergeall` and add tests
Bug from: 981052c9c6
pukkandan 3acf6d3856
[Funimation] Rewrite extractor (See desc) ()
* Support direct `/player/` URL
* Treat the different versions of an episode as different formats of a single video. So `experience_id` can no longer be used as the video `id` and the `episode_id` is used instead. This means that all existing archives will break
* Extractor options `language` and `version` to pre-select them
* Compat option `seperate-video-versions` to fall back to old behavior (including using the old video IDs)

Closes 
pukkandan bc97cdae67
[cleanup] Fix linter and some typos
Related: https://github.com/ytdl-org/youtube-dl/pull/29398
pukkandan 169dbde946
Fixes for `--list` options (See desc)
1. Fix `--list-formats-old`
2. Allow listing with `--quiet`
3. Allow various listings to work together
4. Allow `--print` to work with listing
pukkandan 981052c9c6 Some minor fixes and refactoring (see desc)
* [utils] Fix issues with reversal
* check_formats should catch `DownloadError`, not `ExtractorError`
* Simplify format selectors with `LazyList` and `yield from`
pukkandan 49c258e18d
[youtube] Fix subtitle names for age-gated videos
Related: https://github.com/iv-org/invidious/pull/2205#issuecomment-868680486
pukkandan d3f62c1967
Fix `--throttled-rate` when using `--load-info-json`
pukkandan 5d3a0e794b
Add `--extractor-args` to pass extractor-specific arguments
Adrik 4513a41a72
Process videos when using `--ignore-no-formats-error` ()
Authored by: krichbanana
pukkandan 6033d9808d
Fix `--flat-playlist` when entry has no `ie_key`
pukkandan 51d9739f80
Add option `--throttled-rate` below which video data is re-extracted
Currently only for HTTP downloads

Closes , workaround for https://github.com/ytdl-org/youtube-dl/issues/29326
pukkandan e36d50c5dd
[websockets] Add `WebSocketFragmentFD` ()
Necessary for 

Co-authored by: nao20010128nao, pukkandan
pukkandan f89b3e2d7a
Skip fixup of existing files and add `--fixup force` to force it
pukkandan fd7cfb6444
[cleanup] Refactor fixup
pukkandan 9fea350f0d
Fix id sanitization in filenames
Closes 
pukkandan e858a9d6d3
[EmbedThumbnail] Add compat-option `embed-thumbnail-atomicparsley`
to force use of atomicparsley for embedding thumbnails in mp4

Related: 
pukkandan 551f93885e
Ignore `images` formats from merge
pukkandan 8326b00aab
Allow `images` formats
Necessary for .

* They are identified by `vcodec=acodec='none'`
* These formats show as the worst in `-F`
* Any postprocessor that expects audio/video will be skipped
* `b*` and all related selectors will skip such formats
* This commit also does not add any selector for downloading such formats. They have to be explicitly requested by the `format_id`. Implementation of a selector is left for when  is resolved
pukkandan b0249bcaf0
Expand `--check-formats` to thumbnails
Closes 
pukkandan 21cd8fae49
Use `NamedTemporaryFile` for `--check-formats`
pukkandan 8ba8714880
[EmbedThumbnail] Fix for already downloaded thumbnail
pukkandan 187986a857
Better error handling of syntax errors in `-f`
pukkandan 0181adefc6
[build] Build Windows x86 version with py3.7
and remove redundant tests
Closes 

:ci skip

Co-authored by: pukkandan, shirt-dev
pukkandan 385a27fad1
Improve offset parsing in outtmpl
pukkandan 639f1cea92
Fix `%d` and empty default in outtmpl
Closes 
pukkandan b5c5d84f60
Revert "[build] Build Windows x86 version with py3.8"
This reverts commit aa75e51f99.

See 

This is being reverted instead of modified due to 
pukkandan aa75e51f99
[build] Build Windows x86 version with py3.8
and remove redundant tests
:ci skip

Ao-authored by: pukkandan, shirt-dev
pukkandan 76a264ac9e
Make outtmpl more robust and catch errors early
pukkandan 324ad82006
[utils] Generalize `traverse_dict` to `traverse_obj`
pukkandan bd99f6e648
Add field `original_url` with the user-inputted URL
So that they can be processed by `--parse-metadata` for example

`webpage_url` is the same, but may be modified by the extractor
pukkandan ecb5419149
Make more fields available for `--print` when used with `--flat-playlist`
pukkandan 87ea7dfc04
Fix filename sanitization
Bug from 752cda3880
pukkandan 752cda3880
Fix and refactor `prepare_outtmpl`
The following tests would have failed previously:
%(id)d %(id)r
%(ext)s-%(ext|def)d
%(width|)d
%(id)r %(height)r
%(formats.0)r
%s
felix cc52de4356
[cleanup] Point all shebang to `python3` ()
Authored by: fstirlitz
pukkandan 885cc0b75c
[embedthumbnail] Embed if any thumbnail was downloaded, not just the best
pukkandan ae8f99e648
Remove `None` values from `info.json`
pukkandan 835a1478b4
Write messages to `stderr` when both `quiet` and `verbose`
pukkandan 120fe5134a
Pre-check archive and filters during playlist extraction
This makes `--break-on-existing` much faster.
It also helps `--break-on-reject` if the playlist extractor can extract the relevant fields
pukkandan 56a8fb4f77
Refactor `__process_playlist` using `LazyList`
pukkandan c77495e3a4
[cleanup] `_match_entry`
pukkandan c19bc311cb
[cleanup] Refactor updater
The updater now uses `.update.run_update` and not `.update.update_self`.
Although I don't expect anyone to be using the updater via API, a wrapper `update_self` is provided for compatibility just in case
pukkandan bc516a3f3c
Sanitize and sort playlist thumbnails
Closes 
pukkandan 18e674b4f6
[ffmpeg] Download and merge in a single step if possible
pukkandan 28b0eb0f65
[cleanup] See desc
* Remove struct from `embedthumbnail`
* Use bullet lists in readme where numbered list don't make sense
* Fix error introduced in 9c2b75b561 when `ie_result` is `None`
pukkandan 9c2b75b561
Field `additional_urls` to download additional videos from metadata
pukkandan 4ec82a72bb
Ensure `post_extract` and `pre_process` only run once
Previously, they ran once for each format requested
pukkandan 07cce701de
[cleanup] linter, code formatting and readme
pukkandan ff2751ac9c
[youtube] Always extract `maxresdefault` thumbnail
Fixes: https://github.com/ytdl-org/youtube-dl/issues/29049
pukkandan a06916d98e
[extractor] Add `write_debug` and `get_param`
pukkandan 681de68e9d
Write thumbnail of playlist
Related: https://github.com/ytdl-org/youtube-dl/pull/28872, https://github.com/ytdl-org/youtube-dl/pull/28860
This is slightly different from the above PRs in that this downloads the playlist's thumbnail instead of the uploader's profile picture. But for youtube channel URLs these are the same
pukkandan 7aee40c13c
Fix bug in listing subtitles
Bug introduced by: 2412044c90
pukkandan 2412044c90
Add field `name` for subtitles
Co-authored by: pukkandan, tpikonen

Based on: , https://github.com/ytdl-org/youtube-dl/pull/26112
pukkandan fe346461ff
Fix `--check-formats` when there is network error
pukkandan 53c18592d3
Add option `--print`
Deprecates: `--get-description`, `--get-duration`, `--get-filename`, `--get-format`, `--get-id`, `--get-thumbnail`, `--get-title`, `--get-url`
Closes 
pukkandan e632bce2e4
[options] Refactor callbacks
pukkandan 0760b0a7e2
Standardize `write_debug`
pukkandan d908aa636a
[cleanup] Fix typos
pukkandan 4171221823
Add compat-option `no-attach-infojson`
pukkandan 53ed7066ab
Option `--compat-options` to revert some of yt-dlp's changes
* Deprecates `--list-formats-as-table`, `--list-formats-old`
pukkandan a61f4b287b
Deprecate support for python versions < 3.6
Closes 
pukkandan 41d1cca328
Update to ytdl-commit-a726009
[blinkx] Remove extractor
a726009987
pukkandan 717297545b
Fix `playlist_index` and add `playlist_autonumber` ()
Now `playlist_index` is always the position of the video in the actual playlist and `playlist_autonumber` is the position of the item in the playlist queue
pukkandan e8e738406a
Add experimental option `--check-formats` to test the URLs before format selection
pukkandan e625be0d10
Improve output template internal formatting
* Allow slicing lists/strings using `field.start🔚step`
* A field can also be used as offset like `field1+num+field2`
* A default value can be given using `field|default`
* Capture all format strings and set it to `None` if invalid. This prevents invalid fields from causing errors
pukkandan 848887eb7a
[downloader] Fix `quiet` and `to_stderr`
pukkandan 3158150cb7
[utils] Add `network_exceptions`
pukkandan f59ae58163
Fix number of digits in `%(playlist_index)s`
When used with `--playlist-(items|start|end)`, the number of digits should depend on the last index in the playlist, not number of items
pukkandan 0d1bb027aa
Move option warnings to `YoutubeDL`
Previously, these warnings did not obey `--no-warnings` and did not output colors
pukkandan 4cd0a709aa
Fix `preload_download_archive` writing verbose message to `stdout`
* And move it after all deprecated warnings
pukkandan 58f197b76c
Revert "[core] be able to hand over id and title using url_result"
This reverts commit 0704d2224b.

This is a commit from `youtube-dlc`. It is not clear what the original purpose of this was. It seems to be a way for extractors to pass `title` and `id` through when the entry is processed by another extractor

* But `title` can already be passed through using `url_transparent`
* `id` is never supposed to be passed through since it could cause issues with archiving
pukkandan 598d185db1
Fix case sensitivity of format selector
Bug introduced in f8d4ad9ab0
pukkandan a439a3a45c
Improve output template (see desc)
* Objects can be traversed like `%(field.key1.key2)s`
* A number can be added to the field as `%(field+n)s`
* Deprecates `--autonumber-start`
pukkandan 26e2805c3f
Add option `--skip-playlist-after-errors`
Allows to skip the rest of a playlist after a given number of errors are encountered
pukkandan c32b0aab8a
Improve --sub-langs (see desc)
* Treat `--sub-langs` entries as regex
* `all` can be used to refer to all the subtitles
* the language code can be prefixed with `-` to exclude it
* Deprecates `--all-subs`
Closes 
pukkandan b7da73eb19
Add option `--ignore-no-formats-error`
* Ignores the "no video format" and similar errors
* Experimental - Some extractors may still throw these errors
pukkandan 921b76cab8
Ensure `mergeall` selects best format when multistreams are disabled
pukkandan 56d868dbb7
Allow running some `postprocessors` before actual download
pukkandan e01d6aa435
Fix `mergeall` when requested formats are unavailable
pukkandan 52a8a1e1b9
Option to choose different downloader for different protocols
* Renamed `--external-downloader-args` to `--downloader-args`
* Added `native` as an option for the downloader
* Use similar syntax to `--downloader-args` etc. Eg: `--downloader dash:native --downloader aria2c`
* Deprecated `--hls-prefer-native` and `--hls-prefer-ffmpeg` since the same can now be done with `--downloader "m3u8:native"` and `m3u8:ffmpeg` respectively
* Split `frag_urls` protocol into `m3u8_frag_urls` and `dash_frag_urls`
* Standardize shortening of protocol names with `downloader.shorten_protocol_name`
pukkandan f8d4ad9ab0
Format selector `mergeall` to download and merge all formats
pukkandan eff635394a
Ability to select nth best format
Eg: -f `bv*.2`

* Also cleaned up the related format selection code
pukkandan b050d210df
Fix some typos and linter
pukkandan 143db31d48
Parse metadata from multiple fields
Closes 
pukkandan 498f560638
Ability to load playlist infojson
* If `--no-clean-infojson` is given, the video ids are saved/loaded from in the infojson along with their playlist index
* If a video entry that was not saved is requested, we fallback to using `webpage_url` to re-extract the entries

Related: https://github.com/yt-dlp/yt-dlp/issues/190#issuecomment-804921024
pukkandan 394dcd4486
Write current epoch to infojson when using `--no-clean-infojson`
Closes 
pukkandan 4d971a16b8
Merge webm formats into mkv if thumbnails are to be embedded ()
Co-authored by: Damiano Amatruda <damiano.amatruda@outlook.com>
Damiano Amatruda 3561530776
Parse resolution in info dictionary ()
Damiano Amatruda 4690688658
Use headers and cookies when downloading subtitles ()
pukkandan 23c1a66730 Make sure `post_hook` gets the final filename