Commit Graph

455 Commits (66e7ace17a36ed0f761ae620801e9e27d5c3cb3f)

Author SHA1 Message Date
Sergey M? 5c2266df4b Switch codebase to use sanitized_Request instead of
compat_urllib_request.Request

[downloader/dash] Use sanitized_Request

[downloader/http] Use sanitized_Request

[atresplayer] Use sanitized_Request

[bambuser] Use sanitized_Request

[bliptv] Use sanitized_Request

[brightcove] Use sanitized_Request

[cbs] Use sanitized_Request

[ceskatelevize] Use sanitized_Request

[collegerama] Use sanitized_Request

[extractor/common] Use sanitized_Request

[crunchyroll] Use sanitized_Request

[dailymotion] Use sanitized_Request

[dcn] Use sanitized_Request

[dramafever] Use sanitized_Request

[dumpert] Use sanitized_Request

[eitb] Use sanitized_Request

[escapist] Use sanitized_Request

[everyonesmixtape] Use sanitized_Request

[extremetube] Use sanitized_Request

[facebook] Use sanitized_Request

[fc2] Use sanitized_Request

[flickr] Use sanitized_Request

[4tube] Use sanitized_Request

[gdcvault] Use sanitized_Request

[extractor/generic] Use sanitized_Request

[hearthisat] Use sanitized_Request

[hotnewhiphop] Use sanitized_Request

[hypem] Use sanitized_Request

[iprima] Use sanitized_Request

[ivi] Use sanitized_Request

[keezmovies] Use sanitized_Request

[letv] Use sanitized_Request

[lynda] Use sanitized_Request

[metacafe] Use sanitized_Request

[minhateca] Use sanitized_Request

[miomio] Use sanitized_Request

[meovideo] Use sanitized_Request

[mofosex] Use sanitized_Request

[moniker] Use sanitized_Request

[mooshare] Use sanitized_Request

[movieclips] Use sanitized_Request

[mtv] Use sanitized_Request

[myvideo] Use sanitized_Request

[neteasemusic] Use sanitized_Request

[nfb] Use sanitized_Request

[niconico] Use sanitized_Request

[noco] Use sanitized_Request

[nosvideo] Use sanitized_Request

[novamov] Use sanitized_Request

[nowness] Use sanitized_Request

[nuvid] Use sanitized_Request

[played] Use sanitized_Request

[pluralsight] Use sanitized_Request

[pornhub] Use sanitized_Request

[pornotube] Use sanitized_Request

[primesharetv] Use sanitized_Request

[promptfile] Use sanitized_Request

[qqmusic] Use sanitized_Request

[rtve] Use sanitized_Request

[safari] Use sanitized_Request

[sandia] Use sanitized_Request

[shared] Use sanitized_Request

[sharesix] Use sanitized_Request

[sina] Use sanitized_Request

[smotri] Use sanitized_Request

[sohu] Use sanitized_Request

[spankwire] Use sanitized_Request

[sportdeutschland] Use sanitized_Request

[streamcloud] Use sanitized_Request

[streamcz] Use sanitized_Request

[tapely] Use sanitized_Request

[tube8] Use sanitized_Request

[tubitv] Use sanitized_Request

[twitch] Use sanitized_Request

[twitter] Use sanitized_Request

[udemy] Use sanitized_Request

[vbox7] Use sanitized_Request

[veoh] Use sanitized_Request

[vessel] Use sanitized_Request

[vevo] Use sanitized_Request

[viddler] Use sanitized_Request

[videomega] Use sanitized_Request

[viewvster] Use sanitized_Request

[viki] Use sanitized_Request

[vk] Use sanitized_Request

[vodlocker] Use sanitized_Request

[voicerepublic] Use sanitized_Request

[wistia] Use sanitized_Request

[xfileshare] Use sanitized_Request

[xtube] Use sanitized_Request

[xvideos] Use sanitized_Request

[yandexmusic] Use sanitized_Request

[youku] Use sanitized_Request

[youporn] Use sanitized_Request

[youtube] Use sanitized_Request

[patreon] Use sanitized_Request

[extractor/common] Remove unused import

[nfb] PEP 8
Sergey M․ 67dda51722 Rename compat_urllib_request_Request to sanitized_Request and move to utils
Sergey M․ 82d8a8b6e2 [YoutubeDL] Wrap plain-text URL requests in compat_urllib_request_Request
ashutosh-mishra 17cc153435 Typo fix, found while going through the code.
Sergey M f270cf1a26 Merge pull request from barlik/master
Clarify that automatic subtitles are generated.
hedii 371c3b796c [YoutubeDL] Add playlist finished downloading message (Closes )
Conflicts:
	youtube_dl/YoutubeDL.py
Rastislav Barlik 741dd8ea65 Clarify that automatic subtitles are generated.
It wasn't clear what automatic word mean.
Sergey M․ b3613d36da [YoutubeDL] Sanitize path after output template substitution (Closes )
Yen Chi Hsuan 8b172c2e10 [YoutubeDL] Use DataHandler
Sergey M․ 4bba371644 [YoutubeDL] Autocalculate ext for subtitles when missing
Sergey M․ 6a3f4c3f82 [YoutubeDL] Use custom cookie processor
Sergey M․ 4abf617b9c [YoutubeDL] Ensure bool params always present in downloader
Sergey M․ d3d89c3256 [YoutubeDL] Encode thumbnail filename (Closes )
Sergey M․ 671302b5c0 [YoutubeDL] Remove unused imports
Sergey M․ 51f267d9d4 [YoutubeDL:utils] Move percent encode non-ASCII URLs workaround to http_request and simplify (Closes )
Jaime Marquínez Ferrándiz 232541df44 [YoutubeDL] format spec: correctly handle dashes and other unused operators
'mp4-baseline-16x9' must be handled as a single string, but the '-' was treated as an operator.
Jaime Marquínez Ferrándiz d96d604e53 YoutubeDL: format spec: don't accept a bare '/' ()
Jaime Marquínez Ferrándiz 03950c90f7 Merge remote-tracking branch 'jaimemf/format_spec_groups' (closes )
Jaime Marquínez Ferrándiz 8250c32f49 [YoutubeDL] don't default to 'bestvideo+bestaudio/best' for live videos
Doesn't work currently.
Sergey M․ f72b0a6032 Revert xvid to avi and make docs to be similar to existing external downloader option
Sergey M․ ac9ed061ec Merge branch 'master' of https://github.com/aurium/youtube-dl into aurium-master
Jaime Marquínez Ferrándiz 0a31a35098 [YoutubeDL] format spec: add additional checks for invalid syntax
Jaime Marquínez Ferrándiz bb8e553662 [YoutubeDL] format spec: Do not fail when a filter gives an empty result
For example with 'best[height<40]' we ended getting a 'IndexError: list index out of range'.
Jaime Marquínez Ferrándiz f5f4a27a96 [YoutubeDL] format spec: fix handling of '/' with ','
When using 'bestvideo/best,bestaudio', 'bestvideo/best' must be set as the current_selector (instead of appending it to the selectors), otherwise when it gets the ',' it would append 'None' to the selectors.
Aurélio A. Heckert 1866432db7 Rename --pp-params to --postprocessor-args and access value as super class attribute
Jaime Marquínez Ferrándiz cf2ac6df68 [YoutubeDL] format spec: Fix handling of '+' with '/'
'bestvideo+bestaudio/best' was incorrectly interpreted as 'bestvideo+(bestaudio/best)', so it would fail if 'bestaudio' doesn't exist instead of falling back to 'best'.
Jaime Marquínez Ferrándiz 0130afb76e [YoutubeDL] format spec: allow grouping specifiers with parentheses
Jaime Marquínez Ferrándiz 5acfa126c8 [YoutubeDL] format spec: treat 'all' like a normal specifier
So you can use filters with it, for example 'all[width>=400][width<=600]'.
Jaime Marquínez Ferrándiz 67134eaba1 [YoutubeDL] rework how the format spec is processed
The spec string is processed using 'tokenize.tokenize' to split it in words and operators, the filters are still processed using regular expressions.
This should make easier to allow grouping operators with parens.
Sergey M․ 9603e8a7d9 [YoutubeDL] Handle None width and height similarly to formats
Sergey M. 3c3b4176bd Merge pull request from dstftw/force-generic-extractor
Add --force-generic-extractor
Aurélio A. Heckert 14835de9fb Use shlex.split for --pp-params and update related docs.
Yen Chi Hsuan 0501bfa159 [YoutubeDL] Youku extractor now uses the standard format selection
Shrimadhav U K 180940e02d spelling mistake corrected
acces changed to accessing
Sergey M․ 0072afca8e [YoutubeDL] Remove force_generic_extractor arg from process_ie_result
Sergey M․ 61aa5ba36e [YoutubeDL] Remove global state for force_generic_extractor flag in favor of passing argument
Sergey M․ 9f4323252a [YoutubeDL] Fix for multiple URLs
Sergey M․ d22dec74ff Add `--force-generic-extractor`
For some extractors that are hard to workout a good _VALID_URL we use very vague and unrestrictive ones,
e.g. just allowing anything after hostname and capturing part of URL as id.
If some of these extractors happen to have an video embed of some different hoster or platform
and this scenario was not handled in extractor itself we end up with inability to download this embed
until extractor is fixed to support embed of this kind.
Forcing downloader to use the generic extractor can be a neat temporary solution for this problem.
Example: FiveTV extractor with Tvigle embed - http://www.5-tv.ru/rabota/broadcasts/48/
Sergey M․ a55e36f48d [YoutubeDL] Handle out-of-range timestamps ()
Sergey M․ df15ef8dab [YoutubeDL] Tweak select_format for video only media
Sergey M․ 931bc3c3a7 [YoutubeDL] Do not loose request method information
Sergey M․ 181c7053e3 [YoutubeDL] Make sure all formats have unique format_id
Sergey M․ 71646e4653 [YoutubeDL] Initialize `files_to_delete` (Closes )
Sergey M․ 4a5a898a8f [YoutubeDL] Clarify incompatible formats merge message
When `-f` is not specified it's misleading to see `You have requested ...` as user did not actually request any formats.
Jaime Marquínez Ferrándiz 3884dcf313 YoutubeDL: ignore indexes from 'playlist_items' that are not in the list (fixes )
We ignore them instead of failing to match the behaviour of the 'playliststart' parameter.
Sergey M․ 82245a6de7 [YoutubeDL] Restore filename for thumbnails
Sergey M․ 98b8ec8616 Merge branch 'best-fallback-on-outdated-avconv' of https://github.com/dstftw/youtube-dl into dstftw-best-fallback-on-outdated-avconv
Conflicts:
	youtube_dl/YoutubeDL.py
Yen Chi Hsuan 8e59539752 [postprocessor/embedthumbnail] Use thumbnails downloaded by YoutubeDL
Yen Chi Hsuan 3749e36e9f [YoutubeDL] Fix PEP8 W503
Sergey M․ 97fcf1bbd0 [YoutubeDL] Check if merger can actually merge
Sergey M․ 7fcb605b82 [YoutubeDL] Fallback to `-f best` when merger is outdated
Jaime Marquínez Ferrándiz 95c5534f8e ExecAfterDownloadPP, YoutubeDL: remove unused parameters
Sergey M․ 3799834dcf [YoutubeDL] Do not force bestvideo+bestaudio when outtmpl is stdout ()
Sergey M․ 2699da8041 [YoutubeDL] Improve description file naming
Sergey M․ 98727e123f [YoutubeDL] Improve annotations file naming
Sergey M․ b29e0000e6 [YoutubeDL] Improve JSON info file naming
Sergey M․ 666a9a2b95 [YoutubeDL] Improve audio/video-only file naming
Sergey M․ 38c6902b90 [YoutubeDL] Ensure correct extension is always present for a merged file (Closes )
Sergey M․ cb202fd286 [YoutubeDL] Filter requested info fields on `--load-info` as well
In order to properly handle JSON info files generated by youtube-dl versions prior to 4070b458ec
Jaime Marquínez Ferrándiz df8301fef5 [YoutubeDL] pep8: use 'k not in' instead of 'not k in'
Sergey M․ 4070b458ec [YoutubeDL] Do not write requested info in info JSON file (Closes , closes )
Sergey M․ e01c56f9e1 [YoutubeDL] Generalize best/worst format match behavior
Jaime Marquínez Ferrándiz 8dd5418803 Make 'best' format only match non-DASH formats (closes )
Otherwise it's impossible to only download non-DASH formats, for example `best[height=?480]/best` would download a DASH video if it's the only one with height=480, instead for falling back to the second format specifier.
For audio only urls (soundcloud, bandcamp ...), the best audio will be downloaded as before.
zouhair 5456d78f0c Typo "incompatible" instead of "uncompatible"
Jaime Marquínez Ferrándiz 9f3fa89f7c Remove the --max-quality option
It doesn't work well with 'bestvideo' and 'bestaudio' because they are usually before the max quality.
Format filters should be used instead, they are more flexible and don't require the requested quality to exist for each video.
Yen Chi Hsuan 8be2bdfabd [YoutubeDL] Remove the redundant assignment to old_filename
Caused by commmit 592e97e855
Sergey M․ 6728187ac0 [YoutubeDL] mp3 is compatible with mp4
Sergey M․ 17c8675853 [YoutubeDL] Allow bestvideo+bestaudio/best strategy for ted extractor
Sergey M․ c0dea0a782 [YoutubeDL] Respect explicit `--merge-format-output` for uncompatible formats as well
Sergey M․ 81cd954a51 [YoutubeDL] Merge incompatible formats into mkv ()
Sergey M․ feccf29c87 [YoutubeDL] Make `bestvideo+bestaudio/best` default format when merger is available
Jaime Marquínez Ferrándiz 5b5fbc0867 Detect already merged videos
Without the '--keep-video' option the two files would be downloaded again and even using the option, ffmpeg would be run again, which for some videos can take a long time.
We use a temporary file with ffmpeg so that the final file only exists if it success
Jaime Marquínez Ferrándiz d47aeb2252 FFmpegMergerPP: use the new system for specifying which files can be delete
Jaime Marquínez Ferrándiz 592e97e855 Postprocessors: use a list for the files that can be deleted
We could only know if we had to delete the original file, but this system allows to specify us more files (like subtitles).
Jaime Marquínez Ferrándiz ecc6bd1341 YoutubeDL.post_process: simplify keep_video handling
Since keep_video started as None we always set it to keep_video_wish unless it was None, so in the end keep_video == keep_video_wish. This should have been changed in f3ff1a3696, but I didn't notice it.
Sergey M․ c36a959549 [YoutubeDL] Try to download worst audio+video served by a single file first (Closes )
Jaime Marquínez Ferrándiz 8e4b83b96b Remove check for ssl certs
When it uses a capath instead of a cafile, 'get_ca_certs' or 'cert_store_stats' only returns certificates already used in a connection.
(see )
Sergey M․ a3c7019e06 [YoutubeDL] Check for `get_ca_certs` availability
`get_ca_certs` is not available in python <3.4
Jaime Marquínez Ferrándiz 1a944d8a2a Print a warning if no ssl certificates are loaded
Jaime Marquínez Ferrándiz 70a1165b32 Don't use bare 'except:'
They catch any exception, including KeyboardInterrupt, we don't want to catch it.
Jaime Marquínez Ferrándiz 179d6678b1 Remove the 'stitle' field
A warning has been printed for more than 2 years (since 97cd3afc75)
Kang Hyojun 613b2d9dc6 Fix mistyped docstring indent
Philipp Hagemeister 486dd09e0b [YoutubeDL] Check for bytes instead of unicode output templates ()
Also adapt the embedding examples for those poor souls still using 2.x.
Jaime Marquínez Ferrándiz 3a77719c5a Don't accept '-1' as format, 'all' is clearer
Sergey M․ e5a11a2293 [YoutubeDL] Sanitize path before creating non-existent paths (Closes )
Sergey M․ 1bb5c511a5 [YoutubeDL] Sanitize outtmpl as path
Sergey M․ 7dcad95d4f [YoutubeDL] Sanitize outtmpl as it may contain forbidden characters
Philipp Hagemeister 2461f79d2a [utils] Correct per-request proxy handling
Philipp Hagemeister 91410c9bfa [letv] Add --cn-verification-proxy (Closes )
Philipp Hagemeister c75f0b361a [downloader/external] Add support for custom options (Fixes , closes )
Jaime Marquínez Ferrándiz 31bd39256b --load-info: Use the fileinput module
It automatically handles the '-' filename as stdin
Jaime Marquínez Ferrándiz 003c69a84b Use shutil.get_terminal_size for getting the terminal width if it's available (python >= 3.3)
Jaime Marquínez Ferrándiz 0f2c0d335b [YoutubeDL] Use the InfoExtractor._download_webpage method for getting the subtitles
It handles encodings better, for example for 'http://www.npo.nl/nos-journaal/14-02-2015/POW_00942207'
Jaime Marquínez Ferrándiz bfc993cc91 Merge branch 'subtitles-rework'
(Closes PR )
Jaime Marquínez Ferrándiz b531cfc019 [YoutubeDL] remove compatiblity with the old subtitles system
Jaime Marquínez Ferrándiz 98c70d6fc7 [YoutubeDL] only add normal subtitles to the 'requested_subtitles' field if 'writesubtitles' is True
Jaime Marquínez Ferrándiz 4d17184817 [YoutubeDL] don't set the 'requested_subtitles' without writesubtitles or writeautomaticsub
Philipp Hagemeister 8fb3ac3649 PEP8: W503
Philipp Hagemeister b81a359eb6 [YoutubeDL] Use render_table for format listing
Jaime Marquínez Ferrándiz edab9dbf4d [YoutubeDL] use the 'render_table' function for listing the subtitles