Commit Graph

3175 Commits (069cbece9dba6384f1cc5fcfc7ce562a31af42fc)

Author SHA1 Message Date
github-actions cc0619f62d Release 2023.07.06
Created by: pukkandan

:ci skip all :ci run dl
1 year ago
Simon Sawicki 3121512228
[core] Change how `Cookie` headers are handled
Cookies are now saved and loaded under `cookies` key in the info dict
instead of `http_headers.Cookie`. Cookies passed in headers are
auto-scoped to the input URLs with a warning.

Ref: https://github.com/yt-dlp/yt-dlp/security/advisories/GHSA-v8mc-9377-rwjj

Authored by: Grub4K
1 year ago
coletdjnz f8b4bcc0a7
[core] Prevent `Cookie` leaks on HTTP redirect
Ref: https://github.com/yt-dlp/yt-dlp/security/advisories/GHSA-v8mc-9377-rwjj

Authored by: coletdjnz
1 year ago
bashonly 1ceb657bdd
[fd/external] Scope cookies
- ffmpeg: Calculate cookies from cookiejar and pass with `-cookies` arg instead of `-headers`
- aria2c, curl, wget: Write cookiejar to file and use external FD built-in cookiejar support
- httpie: Calculate cookies from cookiejar instead of `http_headers`
- axel: Calculate cookies from cookiejar and disable http redirection if cookies are passed
    - May break redirects, but axel simply don't have proper cookie support

Ref: https://github.com/yt-dlp/yt-dlp/security/advisories/GHSA-v8mc-9377-rwjj

Authored by: bashonly, coletdjnz
1 year ago
pukkandan ad8902f616
[ie/vidlii] Handle relative URLs
Closes #7480
1 year ago
pukkandan 94ed638a43
[ie/youtube] Avoid false DRM detection (#7396)
Some master manifests contain a mix of DRM and non-DRM formats
1 year ago
pukkandan bc344cd456
[core] Allow extractors to mark formats as potentially DRM (#7396)
This is useful for HLS where detecting whether the format is
actually DRM requires the child manifest to be downloaded.

Makes the error message when using `--test` inconsistent,
but doesn't really matter.
1 year ago
pukkandan 906c0bdcd8
[formats] Fix best fallback for storyboards
Partial fix for #7478
1 year ago
pukkandan 337734d4a8
[cleanup] Misc 1 year ago
pukkandan 47bcd43724
[outtmpl] Pad `playlist_index` etc even when with internal formatting
Closes #7501
1 year ago
pukkandan 662ef1e910
[downloader/http] Avoid infinite loop when no data is received
Closes #7504
1 year ago
coletdjnz 90db9a3c00
[extractor/youtube:stories] Remove (#7459)
YouTube killed them

https://web.archive.org/web/20230630153050/https://support.google.com/youtube/thread/217640760
1 year ago
bashonly 49296437a8
[extractor/twitter] Fix unauthenticated extraction (#7476)
Closes #7473
Authored by: bashonly
1 year ago
bashonly 1cffd621cb
[extractor/twitter:spaces] Fix extraction (#7512)
Closes #7455
Authored by: bashonly
1 year ago
RfadnjdExt 3b7f5300c5
[extractor/googledrive] Fix source format extraction (#7395)
Closes #7344
Authored by: RfadnjdExt
1 year ago
coletdjnz 4dc4d8473c
[extractor/youtube] Ignore incomplete data for comment threads by default (#7475)
For both `--ignore-errors` and `--ignore-errors only_download`. Pass `--no-ignore-errors` to not ignore.

Closes https://github.com/yt-dlp/yt-dlp/issues/7474

Authored by: coletdjnz
1 year ago
c-basalt 8776349ef6
[extractor/vk] VKPlay, VKPlayLive: Add extractors (#7358)
Closes #7107
Authored by: c-basalt
1 year ago
urectanc af1fd12f67
[extractor/stacommu] Add extractors (#7432)
Authored by: urectanc
1 year ago
coletdjnz fcbc9ed760
[extractor/youtube:tab] Support shorts-only playlists (#7425)
Fixes https://github.com/yt-dlp/yt-dlp/issues/7424

Authored by: coletdjnz
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
1 year ago
bashonly a2be9781fb
[extractor/Douyin] Fix extraction from webpage
Closes #7431
Authored by: bashonly
1 year ago
Xiao Han 8f05fbae2a
[extractor/abc] Fix extraction (#7434)
Closes #6433
Authored by: meliber
1 year ago
Aman Salwan 5b4b92769a
[extractor/crunchyroll:music] Fix `_VALID_URL` (#7439)
Closes #7419
Authored by: AmanSal1, rdamas

Co-authored-by: Robert Damas <robert.damas@byom.de>
1 year ago
pukkandan 91302ed349
[utils] clean_podcast_url: Handle protocol in redirect URL
Closes #7430
1 year ago
pukkandan f393bbe724
[extractor/sbs] Python 3.7 compat
Closes #7410
1 year ago
pukkandan 8a8af356e3
[downloader/aria2c] Add `--no-conf`
Closes #7404
1 year ago
pukkandan d949c10c45
[extractor/youtube] Process `post_live` over 2 hours 1 year ago
bashonly ef8509c300
[extractor/kick] Fix `_VALID_URL`
Closes #7384
Authored by: bashonly
1 year ago
nnoboa 5e16cf92eb
[extractor/AdultSwim] Extract subtitles from m3u8 (#7421)
Authored by: nnoboa
Closes #6191
1 year ago
bashonly f0a1ff1181
[extractor/qdance] Add extractor (#7420)
Closes #7385
Authored by: bashonly
1 year ago
pukkandan 58786a10f2
[extractor/youtube] Add extractor-arg `formats`
Closes #7417
1 year ago
pukkandan e59e20744e
Bugfix for b4e0d75848 1 year ago
Simon 89bed01374
[extractor/youtube] Fix comments' `is_favorited` (#7390)
Authored by: bbilly1
Closes #7389
1 year ago
github-actions de4cf77ec1 Release 2023.06.22
Created by: pukkandan

:ci skip all :ci run dl
1 year ago
pukkandan 812cdfa06c
[cleanup] Misc 1 year ago
pukkandan cd810afe2a
[extractor/youtube] Improve nsig function name extraction 1 year ago
pukkandan b4e0d75848
Improve `--download-sections`
* Support negative time-ranges
* Add `*from-url` to obey time-ranges in URL

Closes #7248
1 year ago
Berkan Teber 71dc18fa29
[extractor/youtube] Improve description parsing performance (#7315)
* The parsing is skipped when not needed
* The regex is improved by simulating atomic groups with lookaheads

Authored by: pukkandan, berkanteber
1 year ago
bashonly 98cb1eda7a
[extractor/rheinmaintv] Add extractor (#7311)
Authored by: barthelmannk

Co-authored-by: barthelmannk <81305638+barthelmannk@users.noreply.github.com>
1 year ago
bashonly 774aa09dd6
[extractor/dplay] GlobalCyclingNetworkPlus: Add extractor (#7360)
* Allows `country` API param to be configured with `--xff`/`geo_bypass_country`

Closes #7324
Authored by: bashonly
1 year ago
rexlambert22 f2ff0f6f19
[extractor/motherless] Add gallery support, fix groups (#7211)
Authored by: rexlambert22
1 year ago
pukkandan 5fd8367496
[extractor] Support multiple `_VALID_URL`s (#5812)
Authored by: nixxo
1 year ago
pukkandan 0dff8e4d1e
Indicate `filesize` approximated from `tbr` better 1 year ago
pukkandan 1e75d97db2
[extractor/youtube] Add `ios` to default clients used
* IOS is affected neither by 403 or by nsig so helps mitigate them preemptively
* IOS also has higher bit-rate "premium" formats though they are not labeled as such
1 year ago
pukkandan 81ca451480
[extractor/youtube] Workaround 403 for android formats
Ref: https://github.com/TeamNewPipe/NewPipe/issues/9038#issuecomment-1289756816
1 year ago
Roland Hieber 3f756c8c40
[extractor/nebula] Fix extractor (#7156)
Closes #7017
Authored by: Lamieur, rohieb

Co-authored-by: Lam <github@Lam.pl>
1 year ago
bashonly 7f9c6a63b1
[cleanup] Misc
Authored by: bashonly
1 year ago
OverlordQ db22142f6f
[extractor/dropout] Fix season extraction (#7304)
Authored by: OverlordQ
1 year ago
pukkandan d7cd97e8d8
Fix bug in db3ad8a676
Closes #7367
1 year ago
github-actions d1b2156149 Release 2023.06.21
Created by: pukkandan

:ci skip all :ci run dl
1 year ago
pukkandan 42f2d40b47
Update to ytdl-commit-07af47
[YouTube] Improve fix for ae8ba2c
07af47960f
1 year ago
pukkandan 1619ab3e67
Bugfix for ebe1b4e34f 1 year ago
pukkandan 84078a8b38
[core] Fix `filepath` being copied to underlying format dict
Closes #6536
1 year ago
pukkandan ad54c9130e
[cleanup] Misc
Closes #6288, Closes #7197, Closes #7265, Closes #7353, Closes #5773
Authored by: mikf, freezboltz, pukkandan
1 year ago
Nicolai Dagestad db3ad8a676
Add option `--netrc-cmd` (#6682)
Authored by: NDagestad, pukkandan
Closes #1706
1 year ago
MMM af7585c824
[extractor/tagesschau] Fix single audio urls (#6626)
Authored by: flashdagger
1 year ago
pukkandan 02948a17d9
[update] Do not restart into versions without `--update-to` 1 year ago
pukkandan 424f3bf033
[downloader/fragment] Do not sleep between fragments
Closes #6599
1 year ago
pukkandan ebe1b4e34f
[outtmpl] Fix some minor bugs
Closes #7164
1 year ago
pukkandan a35af4306d
[utils] `strftime_or_none`: Handle negative timestamps
Closes #6706
Authored by pukkandan, dirkf
1 year ago
pukkandan 93b39cdbd9
Add `--compat-option playlist-match-filter`
Closes #6073
1 year ago
pukkandan 97afb093d4
[extractor/youtube] Ignore wrong fps of some formats 1 year ago
pukkandan 2e023649ea
[cookies] Revert compatibility breakage in b38d4c941d 1 year ago
pukkandan 51a07b0dca
[extractor/youtube] Prioritize premium formats
Closes #7283
1 year ago
pukkandan eedda5252c
[utils] `FormatSorter`: Improve `size` and `br`
Closes #1596

Previously, when some formats have accurate size and some approximate,
the ones with accurate size was always prioritized

For formats with known tbr and unknown vbr/abr, we were setting
(vbr=tbr, abr=0) for sorting to work. This is no longer needed.

Authored by pukkandan, u-spec-png
1 year ago
Mozi 5cc09c004b
[extractor/zaiko] ZaikoETicket: Add extractor (#7347)
Authored by: pzhlkj6612
1 year ago
Vladislav 6f69101dc9
[extractor/yappy] YappyProfile: Add extractor (#7346)
Authored by: 7vlad7
1 year ago
garret 81c8b9bdd9
[extractor/nhk] `NhkRadiruLive`: Add extractor (#7332)
Authored by: garret1317
1 year ago
pukkandan 01aba2519a
[jsinterp] Fix global object extraction
Closes #7327
1 year ago
pukkandan 13ff780953
[postprocessor] Print newline for `--progress-template`
Closes #7193
1 year ago
pukkandan ff9b0e071f
[extractor/youtube] Determine audio language using automatic captions 1 year ago
toomyzoom 0a5d7c39e1
[extractor/iwara] Fix authentication (#7137)
Closes #7035, Closes #7207
Authored by: toomyzoom
1 year ago
TxI5 125ffaa173
[extractor/tv4] Fix extractor (#5649)
Closes #5535
Authored by: TxI5, dirkf
1 year ago
foreignBlade f9213f8a2d
[extractor/stripchat] Fix extractor (#7306)
Closes #7305
Authored by: foreignBlade
1 year ago
Jeong, Heon fdd69db389
[extractor/afreecatv] Fix extractor (#6283)
Closes #6133
Authored by: blmarket
1 year ago
Elyse 83465fc410
[extractor/ettutv] Add extractor (#6579)
Closes #6359
Authored by: elyse0
1 year ago
RjY 6daaf21092
[extractor/discogs] Add extractor (#6624)
Authored by: rjy
1 year ago
hoaluvn 7bcd481321
[extractor/urplay] Extract all subtitles (#7309)
Authored by: hoaluvn
1 year ago
bashonly c8561c6d03
[extractor/wrestleuniverse] Fix cookies support
Closes #7298
Authored by: bashonly
1 year ago
Cyberes cab94a0cd8
[extractor/funker530] Add extractor (#7291)
Authored by: Cyberes
1 year ago
c-basalt 345b4c0aed
[extractor/zaiko] Add extractor (#7254)
Closes #7196
Authored by: c-basalt
1 year ago
linsui 8790ea7b25
[extractor/ximalaya] Sort playlist entries (#7292)
Authored by: linsui
1 year ago
puc9 ab6057ec80
[extractor/tiktok] Fix resolution extraction (#7237)
Authored by: puc9
1 year ago
bashonly 9d7fde89a4
[extractor/zee5] Fix extraction of new content (#7280)
Authored by: bashonly
1 year ago
bashonly 1a2eb5bda5
[extractor/odnoklassniki] Fix formats extraction (#7217)
Closes #2959, Closes #4462, Closes #7201
Authored by: bashonly
1 year ago
DataGhost f8ae441501
[extractor/Dumpert] Fix m3u8 and support new URL pattern (#6091)
Authored by: DataGhost, pukkandan
Closes #5032
1 year ago
bashonly b4a252fba8
[jsinterp] Fix division (#7279)
* Fixes nsig decryption for Youtube JS player `8c7583ff`

Authored by: bashonly
1 year ago
bashonly 4f7b11cc1c
[extractor/voot] Fix extractor (#7227)
Closes #6715
Authored by: bashonly
1 year ago
bashonly d1795f4a6a
[extractor/twitter] Add login support (#7258)
Closes #6951
Authored by: bashonly
1 year ago
bashonly 44c0d66442
[extractor/lbry] Extract original quality formats (#7257)
Closes #7251
Authored by: bashonly
1 year ago
coletdjnz 8213ce28a4
[extractor/youtube] Extract `channel_is_verified` (#7213)
Authored by: coletdjnz
1 year ago
pukkandan 14a14335b2
[extractor/youtube] Misc cleanup
Authored by: coletdjnz
1 year ago
stanoarn c2b801fea5
[extractor/rozhlas] `MujRozhlas`: Add extractor (#7129)
Authored by: stanoarn
1 year ago
bashonly 59d9fe0831
[extractor/mgtv] Fix formats extraction (#7234)
Closes #7008
Authored by: bashonly
1 year ago
bashonly ee0ed0338d
[extractor/zdf] Fix formats extraction
Closes #7238, Closes #7240
Authored by: bashonly
1 year ago
bashonly c2a1bdb009
[extractor/tiktok] Extract 1080p adaptive formats (#7228)
Closes #7109
Authored by: bashonly
1 year ago
bashonly 7f8ddebbb5
[extractor/hotstar] Support `/shows/` URLs (#7225)
Closes #6463
Authored by: bashonly
1 year ago
bashonly 7bc9251746
[extractor/shemaroome] Pass `stream_key` header to downloader (#7224)
Closes #7133
Authored by: bashonly
1 year ago
bashonly 4815d35c19
[extractor/sonyliv] Fix login with token (#7223)
Authored by: bashonly
1 year ago
bashonly 97d60ad8cd
[extractor/foxnews] Fix extractors (#7222)
Closes #6050
Authored by: bashonly
1 year ago
bashonly 5ee9a7d6e1
[extractor/sverigesradio] Support slug URLs (#7220)
Closes #7145
Authored by: bashonly
1 year ago
bashonly 971d901d12
[extractor/tencent] Fix fatal metadata extraction (#7219)
Closes #7177
Authored by: bashonly
1 year ago
bashonly 12037d8b0a
[extractor/substack] Fix extraction (#7218)
Closes #7155
Authored by: bashonly
1 year ago
Paul Wise c91ac833ea
[extractor/acast] Support embeds (#7212)
Authored by: pabs3
1 year ago
coletdjnz 2fb35f6004
[extractor/youtube] Support shorter relative time format (#7191)
See: https://github.com/TeamNewPipe/NewPipeExtractor/issues/1067

Authored by: coletdjnz
1 year ago
Jeroen Jacobs 1a7dcca378
[extractor/vrt] Overhaul extractors (#6244)
* Fixes `VrtNU` extractor to work with the VRT MAX site change
* Adapts `VRT`, `Ketnet` and `DagelijkseKost` extractors to the new VRT API
* Removes `Canvas` and `CanvasEen` extractors; the sites and API no longer exist
* Moves all remaining VRT-related extractors into the `vrt` module

Closes #4908
Authored by: jeroenj, bergoid, bashonly

Co-authored-by: bergoid <bergoid@users.noreply.github.com>
Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
1 year ago
Mohamed Al Mehairbi 55ed4ff734
[extractor/DigitalConcertHall] Support films (#7202)
Authored by: ItzMaxTV
Closes #7184
1 year ago
bashonly 01231feb14
[extractor/twitch] Update `_CLIENT_ID` and add extractor-arg (#7200)
Closes #7058, Closes #7183
Authored by: bashonly
1 year ago
Daniel Rich f41b949a2e
[extractor/nhk] Fix API extraction (#7180)
Closes #6992
Authored by: sjthespian, menschel

Co-authored-by: Patrick Menschel <menschel.p@posteo.de>
1 year ago
coletdjnz c35448b7b1
[extractor/youtube] Extract more metadata for comments (#7179)
Adds new comment fields:
* `author_url` - The url to the comment author's page
* `author_is_verified` - Whether the author is verified on the platform
* `is_pinned` - Whether the comment is pinned to the top of the comments

Closes https://github.com/yt-dlp/yt-dlp/issues/5411

Authored by: coletdjnz
1 year ago
CeruleanSky 1c16d9df53
[extractor/twitter:spaces] Add `release_timestamp` (#7186)
Authored by: CeruleanSky
1 year ago
Mohamed Al Mehairbi ecfe47973f
[extractor/elevensports] Add extractor (#7172)
Closes #6737
Authored by: ItzMaxTV
1 year ago
coletdjnz 18f8fba7c8
[extractor/youtube] Fix continuation loop with no comments (#7148)
Deep check the response for incomplete data.

Authored by: coletdjnz
1 year ago
mrscrapy c2502cfed9
[extractor/recurbate] Add extractor (#6297)
Authored by: mrscrapy
1 year ago
bashonly 1fe5bf240e
[extractor/bravotv] Detect DRM (#7171)
Authored by: bashonly
1 year ago
Mohamed Al Mehairbi 26c517b29c
[extractor/crtvg] Add extractor (#7168)
Closes #6609
Authored by: ItzMaxTV
1 year ago
Elyse 6f10cdcf7e
[extractor/bilibili:SpaceVideo] Extract signature (#7149)
Authored by: elyse0
Closes #6956, closes #7081
1 year ago
HobbyistDev 03789976d3
[extractor/europarl] Rewrite extractor (#7114)
Authored by: HobbyistDev
Closes #6396
1 year ago
Mohamed Al Mehairbi dc3c44f349
[extractor/Mzaalo] Add extractor (#7163)
Authored by: ItzMaxTV
1 year ago
Ivan Skodje 937264419f
[extractor/tvplay] Remove outdated domains (#7106)
Closes #3920
Authored by: ivanskodje
1 year ago
Ivan Skodje 372a0f3b9d
Auto-select default format in `-f-` (#7101)
Authored by: ivanskodje, pukkandan
Closes #6720
1 year ago
garret 4cbfa570a1
[extractor/camfm] Add extractors (#7083)
Authored by: garret1317
1 year ago
HobbyistDev 45e87ea106
[extractor/eurosport] Improve `_VALID_URL` (#7076)
Closes #7042
Authored by: HobbyistDev
1 year ago
Florian Albrechtskirchinger dbce5afa6b
[extractor/twitch:vod] Support links from schedule tab (#7071)
Authored by: falbrechtskirchinger
1 year ago
Stefan Lobbenmeier f78eb41e1c
[extractor/ARDBetaMediathek] Add thumbnail (#6890)
Closes #6889
Authored by: StefanLobbenmeier
1 year ago
Matt Broadway b38d4c941d
[cookies] Update for chromium changes (#6897)
Authored by: mbway
1 year ago
hasezoey 489f51279d
[extractor/nekohacker] Add extractor (#7003)
Authored by: hasezoey
1 year ago
JChris246 2d306c03d6
[extractor/rottentomatoes] Fix extractor (#6844)
Closes #6729
Authored by: JChris246
1 year ago
bashonly f6e43d6fa9
[extractor/cbsnews] Overhaul extractors (#6681)
Closes #6565
Authored by: bashonly
1 year ago
bashonly fd5d93f704
Bugfix for b844a3f8b1
[extractor/weverse] Avoid unnecessary duplicate login

Authored by: bashonly
1 year ago
Lesmiscore f8f9250fe2
[extractor/niconico:live] Add extractor (#5764)
Authored by: Lesmiscore
1 year ago
Lesmiscore 3459d3c5af
[extractor/JStream] Add extractor (#6252)
Authored by: Lesmiscore
1 year ago
bashonly c25cac2f8e
[extractor/dacast] Add extractors (#6896)
Closes #6163
Authored by: bashonly
1 year ago
Nam Vu a58182b75a
[cookies] Support custom Safari cookies path (#6783)
Authored by: NextFire
1 year ago
jo-nike 4afb208cf0
[extractor/cbc] Ignore 426 from API (#6781)
Closes #6716
Authored by: jo-nike
1 year ago
ping 5c14b21367
[extractor/idolplus] Add extractor (#6732)
Authored by:  ping
Closes #6246
1 year ago
bepvte 02312c03cf
[extractor/twitch] Support mobile clips (#6699)
Authored by: bepvte
1 year ago
Stefan Borer 94627c5dde
[extractor/playsuisse] Support new url format (#6528)
Authored by: sbor23
1 year ago
Daniel Vogt c6d4b82a8b
[extractor/owncloud] Add extractor (#6533)
Authored by: C0D3D3V
1 year ago
Ha Tien Loi 17d7ca84ea
[extractor/zingmp3] Fix and improve extractors (#6367)
Authored by: hatienl0i261299
1 year ago
Mohit Tokas bfdf144c7e
[extractor/livestream] Support videos with account id (#6324)
Authored by: theperfectpunk
Closes #2225
1 year ago
nixxo c6d3f81a40
[extractor/rai] Rewrite extractors (#5940)
Authored by: nixxo, danog
Closes #5672, closes #6341

Co-authored-by: Daniil Gentili <daniil@daniil.it>
1 year ago
lauren n. liberda aed945e1b9
[extractor/wykop] Add extractors (#6140)
Authored by: selfisekai
1 year ago
JChris246 fc5a7f9b27
[extractor/daftsex] Update domain and embed player url (#5966)
Closes #5881
Authored by: JChris246
1 year ago
lauren n. liberda 738c90a463
[extractor/polskieradio] Improve extractors (#5948)
Authored by: selfisekai
1 year ago
coletdjnz 93e12ed76e
[extractor/youtube] Extract uploader metadata for feed/playlist items
Fixes https://github.com/yt-dlp/yt-dlp/issues/7104

Authored by: coletdjnz
1 year ago
Mohamed Al Mehairbi 6dc00acf0f
[extractor/weyyak] Add extractor (#7124)
Closes #7118
Authored by: ItzMaxTV
1 year ago
coletdjnz daafbf49b3
[core] Support decoding multiple content encodings (#7142)
Authored by: coletdjnz
1 year ago
coletdjnz 3f66b6fe50
[core] Workaround erroneous urllib Windows proxy parsing (#7092)
Convert proxies extracted from windows registry to http for older Python versions.
See: https://github.com/python/cpython/issues/86793

Authored by: coletdjnz
1 year ago
coletdjnz b87e01c123
[cookies] Move `YoutubeDLCookieJar` to cookies module (#7091)
Authored by: coletdjnz
1 year ago
coletdjnz 08916a49c7
[core] Improve HTTP redirect handling (#7094)
Aligns HTTP redirect handling with what browsers commonly do and RFC standards. 

Fixes issues afac4caa7d missed.

Authored by: coletdjnz
1 year ago