Commit Graph

470 Commits (25b921b02c49e54402d03f758179ee37e1cd441a)

Author SHA1 Message Date
pukkandan ac66811112
[compat] Remove more functions
Removing any more will require changes to a large number of extractors
pukkandan bc40160883
Fix `section_end` of clips
Closes 
pukkandan 1ac4fd80c8
Fix playlist error handling
Bug in 7e88d7d78f
pukkandan eff4275925
Add deprecation warning for Py3.6
See: https://github.com/yt-dlp/yt-dlp/issues/3764
pukkandan 3975b4d2e8
Allow extractors to specify section_start/end for clips
pukkandan 7b2c3f47c6
[cleanup] Misc
pukkandan 8aa0e7cd96
[docs] Improvements
pukkandan bf1824b391
[cleanup] Deprecate `YoutubeDL.parse_outtmpl`
pukkandan 7e9a612585
Add option `--lazy-playlist` to process entries as they are received
pukkandan 7e88d7d78f
Add slicing notation to `--playlist-items`
* Adds support for negative indices and step
* Add `-I` as alias for `--playlist-index`
* Deprecates `--playlist-start`, `--playlist-end`, `--playlist-reverse`, `--no-playlist-reverse`

Closes , Closes 
pukkandan f0c9fb9682
[utils] `Popen`: Refactor to use contextmanager
Fixes https://github.com/yt-dlp/yt-dlp/issues/3531#issuecomment-1156223597
pukkandan 560738f34d
[extractor] Import `_ALL_CLASSES` lazily
This significantly speeds up `import yt_dlp` in the absence of `lazy_extractors`
pukkandan fccf90e7f3
Fix bug in 56ba69e4c9
pukkandan 64fa820ccf
[cleanup] Misc fixes (see desc)
* [tvver] Fix bug in 6837633a4a - Closes 
* [rumble] Fix tests - Closes 
* [make] Remove `cat` abuse - Closes 
* [make] Revert  - Closes 
* [utils] Improve `get_elements_by_class` - Closes 
* [utils] Inherit `Namespace` from `types.SimpleNamespace`
* [utils] Use `re.fullmatch` for matching filters
* [jsinterp] Handle quotes in `_separate`
* [make_readme] Allow overshooting last line

Authored by: pukkandan, kwconder, MrRawes, Lesmiscore
pukkandan 56ba69e4c9
[cleanup] Misc fixes
Closes 
pukkandan 5ec1b6b716
Add option `--download-sections` to download video partially
Closes , Closes 
pukkandan 1890fc6389
[cleanup] Misc fixes
Cherry-picks from: , 
Related: , https://github.com/yt-dlp/yt-dlp/issues/1839#issuecomment-1140313836
Authored by: pukkandan, flashdagger, gamer191
pukkandan 0bf9dc1e35
Fix bug in 8a82af3511
pukkandan 8a82af3511
[cleanup] Misc fixes and cleanup
Closes , Closes , Closes 
pukkandan c487cf0010
[cleanup] Misc
pukkandan 415f8d51a8
Ensure pre-processor errors do not block video download
Closes 
pukkandan ca6d59d2c1
Fix `--simulate --max-downloads`
Bug in c3e6ffba53
Closes 
pukkandan 53973b4d2c
[utils] Fix bug in 0b9c08b47b
* Cache of `supports_terminal_sequences` must be reset after enabling VT mode
* and move `windows_enable_vt_mode` to utils to avoid cyclic imports
pukkandan 23326151c4
Add option --retry-sleep ()
Closes 
pukkandan 21633673c3
[cleanup] Minor fixes
pukkandan 7896214c42
Bugfix for 591bb9d355
Closes 
pukkandan 5792c950bf
[compat] Implement `compat.imghdr`
Python 3.11 deprecates `imghdr` module
pukkandan 591bb9d355
Fix color in `-q -F`
and convert `ydl._out_files`/`ydl._allow_colors` to `Namespace`

Closes 
pukkandan 490110c543
`--max-downloads` should obey `--break-per-input`
pukkandan 2414649192
[cleanup] Misc cleanup
pukkandan 494f52308b
[FixupM3u8] Obey `--hls-prefer-mpegts`
Closes 
pukkandan 3a408f9d19
Show name of downloader in verbose log
Closes 
pukkandan d76fa1f3d4
[cookies] Allow `cookiefile` to be a text stream
Closes 
pukkandan 0f06bcd759
[cleanup] Minor fixes (See desc)
* [youtube] Fix `--youtube-skip-dash-manifest`
* [build] Use `$()` in `Makefile`. Closes 
* Fix bug in 385ffb467b
* Fix bug in 43d7f5a5d0
* [cleanup] Remove unnecessary `utf-8` from `str.encode`/`bytes.decode`
* [utils] LazyList: Expose unnecessarily "protected" attributes
and other minor cleanup
pukkandan d7a1aa00c6
Run `FFmpegFixupM3u8PP` for live-streams if needed
Closes 
pukkandan d4736fdb43
Remove warning for videos with an empty title
coletdev bb58c9ed5c
Add support for SSL client certificate authentication ()
Adds `--client-certificate`, `--client-certificate-key`, `--client-certificate-password`

Authored-by: coletdjnz
Co-authored-by: df <fieldhouse@gmx.net>
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
pukkandan 3fe75fdc80
[cleanup] Misc fixes (see desc)
* Do not warn when fixup is skipped for existing file
* [fragment] Fix `--skip-unavailable-fragments` for HTTP Errors
* [utils] write_string: Fix bug in 59f943cd50
* [utils] parse_codecs: Subtitle codec is generally referred to as `scodec`. https://github.com/yt-dlp/yt-dlp/pull/2174#discussion_r790156048
* [docs] Remove note about permissions. Closes 
pukkandan 43d7f5a5d0
[EmbedThumbnail] Do not obey `-k`
pukkandan 1d485a1a79
[cleanup] Misc fixes
Closes , https://github.com/yt-dlp/yt-dlp/issues/3514#issuecomment-1105944364
pukkandan 0a41f331cc
[doc] Minor improvements
Closes , Closes 
pukkandan 492272fed6
`--match-filter -` to interactively ask for each video
pukkandan 59f943cd50
[utils] `write_string`: Workaround newline issue in `conhost`
On windows `conhost`, when `WINDOWS_VT_MODE` is enabled, `\n` is not
actually sent if the window is exactly the length of printed line,
and the line does not end with a white-space character. So the
line-break disappears when resizing the window.

Fixes 
pukkandan 0a5a191a2a
Improve `--clean-infojson`
It should not removes fields that may be needed for `--load-infojson`.
Eg: `_ffmpeg_args`, `_has_drm`
pukkandan 9b8ee23b99
[dependencies] Create module with all dependency imports
pukkandan 19a0394044
[cleanup] Misc cleanup and refactor ()
pukkandan e5a998f368
[cleanup] Misc cleanup ()
Authored by: fstirlitz, pukkandan
pukkandan f82711587c
[cleanup] Sort imports
Using https://github.com/PyCQA/isort

    isort -m VERTICAL_HANGING_INDENT --py 36 -l 80 --rr -n --tc .
pukkandan 86e5f3ed2e
[cleanup] Upgrade syntax
Using https://github.com/asottile/pyupgrade

1. `__future__` imports and `coding: utf-8` were removed
2. Files were rewritten with `pyupgrade --py36-plus --keep-percent-format`
3. f-strings were cherry-picked from `pyupgrade --py36-plus`

Extractors are left untouched (except removing header) to avoid unnecessary merge conflicts
pukkandan f9934b9614
[cleanup] Mark some compat variables for removal ()
Authored by fstirlitz, pukkandan
felix cfb0511d82
[cleanup] Remove unused code paths ()
Notes:

* `_windows_write_string`: Fixed in 3.6
  * https://bugs.python.org/issue1602
  * PEP: https://www.python.org/dev/peps/pep-0528

* Windows UTF-8 fix: Fixed in 3.3
  * https://bugs.python.org/issue13216

* `__loader__`: is always present in 3.3+
  * https://bugs.python.org/issue14646

* `workaround_optparse_bug9161`: Fixed in 2.7
  * https://bugs.python.org/issue9161

Authored by: fstirlitz
pukkandan 97ec5bc550
[cookies] Report progress when importing cookies
pukkandan 4abea8ca0a
[utils] `sanitize_path`: Fix when path is empty string
pukkandan d8a58ddce7
De-prioritize automatic-subtitles when no `--sub-lang` is given
Closes 
pukkandan 316f2650f8
Ignore `mhtml` formats from `-f mergeall`
Closes 
pukkandan a44ca5a470
[cleanup] Misc fixes
Closes https://github.com/yt-dlp/yt-dlp/pull/3213, Closes https://github.com/yt-dlp/yt-dlp/pull/3117

Related: https://github.com/yt-dlp/yt-dlp/issues/3146#issuecomment-1077323114, https://github.com/yt-dlp/yt-dlp/pull/3277#discussion_r841019671, a825ffbffa (commitcomment-68538986), https://github.com/yt-dlp/yt-dlp/issues/2360, 5fa3c9a88f (r70393519), 5fa3c9a88f (r70393254)
pukkandan 85e801a9db
Fallback to video-only format when selecting by extension
Closes 
pukkandan 5127e92a94
Fix filepath sanitization in `--print-to-file`
pukkandan cb96c5be70
Fix `--no-overwrite` for playlist infojson
Fixes: https://github.com/yt-dlp/yt-dlp/issues/1467#issuecomment-1079922971
pukkandan 90137ca4be
[utils] Add `filter_dict`
pukkandan c0b6e5c74d
Show warning when all media formats have DRM
Related: 
pukkandan 5c3895fff1
[outtmpl] Limit changes during sanitization
Closes 
pukkandan 34baa9fdf0
[outtmpl] Fix replacement/default when used with alternate
pukkandan 6db9c4d57d
Ignore format-specific fields in initial pass of `--match-filter`
Closes 
pukkandan ae6a1b9585
[docs] Minor improvements
Closes , Closes , Closes 
pukkandan c70c418d33
Fix `--abort-on-error` for subtitles
Closes 
coletdev d5820461e8
Use certificates from `certifi` if installed ()
Fixes  and most `CERTIFICATE_VERIFY_FAILED` issues

Authored by: coletdjnz
pukkandan 0c14d66ad9
Fix `autonumber`
Bug in 09b49e1f68
pukkandan 16c620bc55
Handle float in `--wait-for-video`
Closes 
pukkandan 592b748582
[cleanup] Minor cleanup
Closes 
pukkandan cf4f42cb97
Protect stdout from unexpected progress and console-title
Closes 
MMM 10331a2672
Fix `--print` with `--ignore-no-formats` when url is `None` ()
Authored by: flashdagger
pukkandan 08d30158ec
[cleanup, docs] Misc cleanup
Closes , closes , closes , closes 
coletdev 4390d5ec12
Add brotli content-encoding support ()
Authored by: coletdjnz
pukkandan 50e93e03a7
Update to ytdl-commit-6508688
Make default upload_/release_date a compat_str
6508688e88

Except:
* "[NDR] Overhaul NDR and NJoy extractors" https://github.com/ytdl-org/youtube-dl/pull/30531
    - 01824d275b
    - 39a98b09a2
    - f0a05a55c2
    - 4186e81777
pukkandan 8b7539d27c
Implement `--add-header` without modifying `std_headers`
Closes , 
pukkandan 2a938746f3
Fix verbose log when stdout/stderr encoding is `None`
See: 5c10453827
pukkandan 319b6059d2
Better error message when no --live-from-start format
pukkandan 4c3f8c3fb6
Handle negative duration from extractor
Closes 
Lesmiscore (Naoya Ozaki) 03f830040a
[YoutubeDL] Fill more fields for playlists ()
pukkandan 09b49e1f68
Add pre-processor stage `after_filter`
* Move `_match_entry` and `post_extract` to `process_video_result`. It is also left in `process_info` for API compat
* `--list-...` options and `--force-write-archive` now obey filtering options
* Move `SponsorBlockPP` to `after_filter`. Closes https://github.com/yt-dlp/yt-dlp/issues/2536
* Reverts 4ec82a72bb since this commit addresses the issue it was solving
pukkandan 60f3e99592
Tolerate failure to `--write-link` due to unknown URL
Closes 
pukkandan 8d93e69d67
Create necessary directories for `--print-to-file`
Closes 
pukkandan 3aa915400d
Fix `-all` for `--sub-langs`
Closes 
pukkandan 5c10453827
Fix for when stdout/stderr encoding is None
Closes 
pukkandan e8969bda94
Obey `--abort-on-error` for "ffmpeg not installed"
Closes 
pukkandan 57ebfca39b
Set `webpage_url_...` from `webpage_url` and not input URL
Closes 
Lukas Fink 1ce9a3cb49
Add regex operator and quoting to format filters ()
Closes  
Authored by: lukasfink1
pukkandan 297e9952b6
[extractor] Allow `http_headers` to be specified for `thumbnails`
pukkandan 455a15e2dc
[cleanup,docs] Minor fixes
Closes , 
pukkandan 6a0546e313
[outtmpl] Handle hard-coded file extension better
When we know that the user-provided extension is the correct final one,
replace it with intermediate extension during download
pukkandan fc08bdd6ab
[extractor] Allow non-fatal `title` extraction
pukkandan 88f23a18e0
[docs,cleanup] Fix linter and misc cleanup
Closes 
pukkandan bb66c24797
Add option `--print-to-file`
Closes 
pukkandan 6a5a30f9e2
Ensure `_type` is present in `info.json`
Closes 
pukkandan d37707bda4
Fix/improve `InAdvancePagedList`
pukkandan 6970b6005e
[cleanup] Minor fixes
Closes 
pukkandan 0bfc53d05c
List playlist thumbnails in `--list-thumbnails`
xtkoba f81c62a6a4
Add option --legacy-server-connect ()
to allow HTTPS connection to servers that do not support RFC 5746 secure renegotiation

Authored by: xtkoba
MinePlayersPE 426764371f
[iq.com] Add extractors ()
Closes 
Authored by: MinePlayersPE
pukkandan 3b603dbdf1
Add option `--concat-playlist`
Closes , related: 
pukkandan ed8d87f911
[cleanup, docs] Minor fixes
Closes 
pukkandan e04938ab88
Check for existing thumbnail/subtitle in final directory
Closes 
pukkandan dc5f409cdc
Fix typo in ed5835b451
pukkandan 8130779db6
Allow listing formats, thumbnails, subtitles using `--print` ()
Closes 
Authored by: pukkandan, Zirro
pukkandan ed5835b451
Allow `--print` to be run at any post-processing stage
coletdjnz f0d785d3ed
[youtube:tab] Extract more playlist metadata ()
* Add fields modified_date, modified_timestamp
* Add field playlist_count
* [youtube:tab] Extract view_count, playlist_count, modified_date

Authored by: coletdjnz, pukkandan
pukkandan 9e907ebddf
[cleanup] Misc cleanup
pukkandan b62fa6d75f
Fix `-s --ignore-no-formats --force-write-archive`
Bug in a13e684813
pukkandan be72c62480
Fix recursion error in f46e2f9d92
Closes 
pukkandan a13e684813
Write `download_archive` only after all formats are downloaded
Closes 
pukkandan f46e2f9d92
Add key `requested_downloads` in the root `info_dict`
pukkandan 9c906919ae
Add field `video_autonumber`
Closes 
pukkandan 6020e05d23
Raise error if subtitle download fails
Closes 
pukkandan ebed8b3732
Add more post-processing stages
playlist = After entire playlist
after_video = After downloading all formats of a video
pukkandan 1e43a6f733
Allow `--exec` to be run at any post-processing stage
Deprecates `--exec-before-download`
pukkandan ca30f449a1
Add `--print playlist:` to print fields per playlist
pukkandan 4c922dd3fc
Fix live title for multiple formats
pukkandan abbeeebc4c
[outtmpl] Alternate form for `D` and fix suffix's case
Fixes: https://github.com/yt-dlp/yt-dlp/issues/2085#issuecomment-1002247689, https://github.com/yt-dlp/yt-dlp/pull/2132/files#r775729811
pukkandan ceb98323f2
Don't treat empty containers as `None` in `sanitize_info`
Matt Broadway f59f5ef8b6
[cookies] Support other keyrings ()
Authored by: mbway
pukkandan 6da22e7d4f
Avoid recursion error when re-extracting info
Benedikt Wildenhain 4b4b7f746c
[OpenCast] Add extractors ()
Original PR: https://github.com/ytdl-org/youtube-dl/pull/26934
Authored by: bwildenhain, C0D3D3V
pukkandan 37893bb0c9
[outtmpl] Change filename sanitization type to `S`
`F` is already used for float!
Bug in e0fd95737d
Emanuel Hoogeveen 205a0654c0
Add option `--file-access-retries` ()
Closes 
Authored by: ehoogeveen-medweb
pukkandan b69fd25c25
[cleanup] Misc cleanup
Closes     
pukkandan e0fd95737d
[outtmpl] Add alternate forms `F`, `D`
and improve `id` detection

F = sanitize as filename (# = restricted)
D = add Decimal suffixes

Closes , 2081
pukkandan fa9f30b802
Add interactive format selection with `-f -`
Closes 
The Hatsune Daishi adbc4ec4bb
[dash,youtube] Download live from start to end ()
* Add option `--live-from-start` to enable downloading live videos from start
* Add key `is_from_start` in formats to identify formats (of live videos) that downloads from start
* [dash] Create protocol `http_dash_segments_generator` that allows a function to be passed instead of fragments
* [fragment] Allow multiple live dash formats to download simultaneously
* [youtube] Implement fragment re-fetching for the live dash formats
* [youtube] Re-extract dash manifest every 5 hours (manifest expires in 6hrs)
* [postprocessor/ffmpeg] Add `FFmpegFixupDuplicateMoovPP` to fixup duplicated moov atoms

Known issue: Ctrl+C doesn't work on Windows when downloading multiple formats

Closes 
Authored by: nao20010128nao, pukkandan
Unit 193 e75bb0d6c3
[cleanup] Fix some typos ()
Authored by: unit193
pukkandan dd0228ce1f
Remove known invalid thumbnails from `info_dict`
Related: https://github.com/yt-dlp/yt-dlp/issues/980#issuecomment-997396821
PilzAdam e978789f0f
[outtmpl] Add operator `&` for replacement text ()
Authored by: PilzAdam
pukkandan 39ca3b5c7f
[extractor] Standardize `_live_title`
pukkandan 0bb322b9c0
Add field `webpage_url_domain`
Closes 
pukkandan 5bfc8bee5a
Fix PostProcessor hooks not registered for some PPs
Closes 
pukkandan e3c7d49571
[compat] Suppress errors in enabling VT mode
Closes 
pukkandan b5475f1145
Pre-process when using `--flat-playlist`
pukkandan 38d79fd16c
Use `parse_duration` for `--wait-for-video`
and some minor fix
pukkandan 3efb96a6d1
Fix control characters being printed to `--console-title`
Closes 
pukkandan 91f071af60
Release 2021.12.01
pukkandan 5c22c63da3
Fix `--trim-filename` when filename has `.`
Closes 
pukkandan ee8dd27a73
[cleanup] Add deprecation warnings
pukkandan f304da8a29
[cleanup] Misc cleanup
Closes , closes 
pukkandan 36eaf3039a
[build] Save Git HEAD at release alongside version info
pukkandan f2ebc5c7be
Option `--wait-for-video` to wait for scheduled streams
pukkandan b222c27145
Option `--break-per-input` to apply --break-on... to each input URL
pukkandan 5e5be0c0b2
Fix --break-on-archive when pre-checking
pukkandan 7578d77d8c
[downloader] Add colors to download progress
pukkandan 75689fe59b
Ensure directory exists when checking formats
pukkandan e04b003e64
[FixupM3u8] Fixup MPEG-TS in MP4 container
Closes , https://github.com/ytdl-org/youtube-dl/issues/26410