Commit Graph

308 Commits (78593e294cc4623108d17b1b0b2d26d507953006)

Author SHA1 Message Date
Sergey M․ 4a69fa04e0
[downloader/dash] Abort download immediately after giving up on some fragment
Sergey M․ 2e99cd30c3
[downloader/dash:hls] Report exact fragment error on retry
Sergey M․ 25afc2a783
[downloader/dash:hls] Respect --fragment-retries and --skip-unavailable-fragments (Closes , closes )
Sergey M․ 9603b66012
Introduce --skip-unavailable-fragments
Yen Chi Hsuan 6c9b71bc08
[downloader/external] Recommend --hls-prefer-native for SOCKS users
Related: 
Remita Amine b8079a40bc [turner] fix secure m3u8 formats downloading
Yen Chi Hsuan 20bad91d76
[downloader/external] Clarify that ffmpeg doesn't support SOCKS
Ref: 
Remita Amine 07ea9c9b05 [downloader/hls] fill IV with zeros for IVs shorter than 16-octet
Remita Amine aaf44a2f47 [uplynk] Add new extractor
Remita Amine acfccacad5 [downloader/external:curl] Clarify why CurlFD should not capture stderr
Remita Amine 98e698f1ff [external/curl] respect more downloader options and display progress
Sergey M․ 1ad6b891b2
Add more checks for --min/max-sleep-interval arguments and use more idiomatic naming
singh-pratyush96 065bc35489
Add --max-sleep-interval (Closes )
Yen Chi Hsuan 477b7a8474
[downloader/f4m] Fix for Rai live streams
Sergey M․ 8369a4fe76
[downloader/hls] Simplify and carry long lines
remitamine e154c65128 [downloader/hls] Add support for AES-128 encrypted segments in hlsnative downloader
Sergey M․ e69f9f5d68
[downloader/external] Decode error string before writing to stderr
Yen Chi Hsuan 50ce1c331c
[downloader/external] Add another env for proxies in ffmpeg/avconv
Related sources:
https://git.libav.org/?p=libav.git;a=blob;f=libavformat/http.c;h=8fe8d11e1edfdbb04a8726db2c49cfef3f572aac;hb=HEAD#l152
https://git.libav.org/?p=libav.git;a=blob;f=libavformat/tls.c;h=fab243e93e20034e88e619188c13a44a5d8ccdb9;hb=HEAD#l63
https://github.com/FFmpeg/FFmpeg/blob/f8e89d8/libavformat/http.c#L191
https://github.com/FFmpeg/FFmpeg/blob/f8e89d8/libavformat/tls.c#L92
Sergey M․ 633b444fd2
[downloader/hls] Correct comment on twitch vods
Sergey M․ 51c4d85ce7
[downloader/hls] PEP 8
Sergey M․ 1e236d7e23
[downloader/hls] Do not rely on EXT-X-PLAYLIST-TYPE:EVENT
Sergey M․ 2615fa7584
[downloader/f4m] Simply select format when it's the only one
Yen Chi Hsuan 1b405bb47d
[downloader/f4m] Tolerate truncate segments when testing
Replaces 

Fixes  and test_Bloomberg partially
Yen Chi Hsuan edaa23f822
[compat] Rename struct_(un)pack to compat_struct_(un)pack
Yen Chi Hsuan dab0daeeb0
[utils,compat] Move struct_pack and struct_unpack to compat.py
Sergey M․ 2937590e8b
[downloader/hls] PEP 8
Sergey M․ e62d9c5caa
[downloader/external] Call ffmpeg with with HTTP_PROXY env variable set ()
Sergey M․ 6104cc2985
[downloader/hls] Add event media playlists to unsupported features of hlsnative
Sergey M․ c15c47d19b
[downloader/hls] Remove EXT-X-MEDIA-SEQUENCE from unsupported features for hlsnative
Sergey M eb785b856f Merge pull request from dstftw/hls-native-to-ffmpeg-delegation
[downloader/hls] Delegate extraction to ffmpeg when unsupported features detected
Sergey M․ f5436c5d9e
[downloader/external] Add temp fix ffmpeg m3u8 downloads (Closes )
Sergey M․ 0d66bd0eab
[downloader/hls] Delegate extraction to ffmpeg when unsupported features detected
Sergey M․ bf09af3acb
Add --hls-prefer-ffmpeg
remitamine d868f43c58 [ffmpeg] check for - file name in _ffmpeg_filename_argument
remitamine ce599d5a7e [downloader/external] enable piping for FFmpegFD(closes )
Yen Chi Hsuan f5d8743e0a
[downloader/rtsp] Print the command
Sergey M․ f7df343b4a [downloader/f4m] Extract routine for removing unsupported encrypted media
Sergey M․ 16a8b7986b [downloader/fragment] Document fragment_retries
Sergey M․ 617e58d850 [downloader/{common,fragment}] Fix total retries reporting on python 2.6
Sergey M․ e33baba0dd [downloader/dash] Add fragment retry capability
YouTube may often return 404 HTTP error for a fragment causing the
whole download to fail. However if the same fragment is immediately
retried with the same request data this usually succeeds (1-2 attemps
is usually enough) thus allowing to download the whole file successfully.
So, we will retry all fragments that fail with 404 HTTP error for now.
Sergey M․ 721f26b821 [downloader/fragment] Add report_retry_fragment
remitamine 4230c4894d [external/downloader] fix rtmp downloading using FFmpegFD
remitamine 2e7e561c1d Merge pull request from remitamine/ffmpegfd
[downloader/external] Add FFmpegFD
remitamine d8515fd41c [downloader/external] pass configuration args to ffmpeg
remitamine 694c47b261 [external/downloader] don't pass -t and -ss to ffmpeg
remitamine 77dea16ac8 [downloader/external] check for ffmpeg availablity when it used for m3u8 download
remitamine 6ae27bed01 [download/external] move the check for multiple selected formats to get_suitable_downloader
remitamine da1973a038 [extractor/__init__] disable time range downloading
remitamine be24916a7f [downloader/rtsp] Add rtsp and mms downloader
remitamine 2cb99ebbd0 [downloader/external] add can_download mathod for checking downloader availibilty and support
remitamine 91ee320bfa [downloader/external] wrap available_opt in a list
remitamine 634415ca17 [downloader/external] skip FFmpegFD when requesting multiple formats
remitamine a755f82549 [ffmpeg] convert format ext to ffmpeg output formats codes
Sergey M․ 1b5284b13f [downloader/fragment] Make speed more smooth
At the beginning of every segment there was a drop to Unknown speed due to timeslice being too small to calculate speed.
Now last speed from the previous fragment is used.
Yen Chi Hsuan e9c0cdd389 [jython] Introduce compat_os_name
os.name is always 'java' on Jython
remitamine f34294fa0c [downloader/external:ffmpegfd] check for None value of start_time
remitamine 99cbe98ce8 [downloader/external] check for external downloaders availability
remitamine 12b84ac8c1 [downloader/external] Add FFmpegFD(fixes )
- replace HlsFD and RtspFD
- add basic support for downloading part of the video or audio
Sergey M․ 611c1dd96e [refactor] Single quotes consistency
Sergey M․ d800609c62 [refactor] Do not specify redundant None as second argument in dict.get()
Sergey M․ c78c9cd10d [downloader/dash] PEP 8
remitamine c43fe0268c [downloader/dash] Implement dashsegments fd in terms of fragment fd
Sergey M․ 56401e1e5f [downloader/hls] Do not send 'q' to ffmpeg on Windows (Closes )
remitamine a1a22572fb [downloader/dash] make initialization_url optional
Sergey M․ b8c9926c0a [downloader/f4m] Do not update fragment list while test
Sergey M․ 2c2f1efdcd [downloader/fragment] Remove superfluous whitespace
Sergey M․ 09104e9930 [downloader/f4m] Add live stream flag to context
Now download progress for f4m livestreams is reported correctly
Sergey M․ 5fa1702ca6 [downloader/fragment] Do not report total bytes estimation and eta for live streams
Sergey M․ 8ab3fe81d8 [downloader/f4m] Prefer bootstrap url attribute over inline bootstrap info
Jaime Marquínez Ferrándiz 7d106a65ca Add --hls-use-mpegts option
When using the mpegts container hls vidoes can be played while being downloaded (useful if you are recording a live stream).
VLC and mpv play them file, but QuickTime doesn't.
Jaime Marquínez Ferrándiz 32d77eeb04 [downloader/common] report_retry: Don't crash when retries is infinite (fixes )
Jaime Marquínez Ferrándiz 9d90e7de03 [downloader/hls] Ask ffmpeg to quit when interrupting youtube-dl with 'Ctrl+C' ()
Otherwise the mp4 file can't be played.
Sergey M․ b83b782dc4 [downloader/fragment] Move helper data to context dict
Sergey M․ 709185a264 [downloader/fragment] More smooth calculations
`downloaded_bytes` is now updated on each fragment progress hook invocation
Sergey M․ 9cb1a06b6c [downloader/fragment] Remove unused code and fix zero division error
Jaime Marquínez Ferrándiz 3c91e41614 [downloader/fragment] Don't fail if the 'Content-Length' header is missing
In some dailymotion videos (like http://www.dailymotion.com/video/x3k0dtv from ) the segments URLs don't have the 'Content-Length' header and HttpFD sets the 'totat_bytes' field to None, so we also use '0' in that case (since we do different math operations with it).
Jaime Marquínez Ferrándiz 3f17c357d9 [downloader/hls] Don't let ffmpeg read from stdin ()
If you run 'while read aurl ; do youtube-dl "${aurl}"; done < path_to_batch_file'  (batch_file contains one url per line that uses the hls downloader) each call to youtube-dl consumed some characters and 'read' would assing to 'aurl' a non valid url

(This is the same problem that was fixed for the ffmpeg postprocessors in cffcbc02de)
Sergey M․ 9b9c5355e4 Rename error_to_str to error_to_compat_str
Sergey M․ 7f8b271465 Properly convert errors to strings
Sergey M․ 4db43567e8 [downloader/f4m] Decode manifest before fixing
Sergey M․ 17b786ae73 [downloader/f4m] Fix malformed manifests (Closes )
Yen Chi Hsuan 94e8c80473 [downloader/hls] Respect Youtubedl-* headers
Yen Chi Hsuan 9d0e366880 [downloader/hls] Remove Accept-encoding from headers passed to ffmpeg
Fails for Youtube Gaming live streams ()
Sergey M? 5c2266df4b Switch codebase to use sanitized_Request instead of
compat_urllib_request.Request

[downloader/dash] Use sanitized_Request

[downloader/http] Use sanitized_Request

[atresplayer] Use sanitized_Request

[bambuser] Use sanitized_Request

[bliptv] Use sanitized_Request

[brightcove] Use sanitized_Request

[cbs] Use sanitized_Request

[ceskatelevize] Use sanitized_Request

[collegerama] Use sanitized_Request

[extractor/common] Use sanitized_Request

[crunchyroll] Use sanitized_Request

[dailymotion] Use sanitized_Request

[dcn] Use sanitized_Request

[dramafever] Use sanitized_Request

[dumpert] Use sanitized_Request

[eitb] Use sanitized_Request

[escapist] Use sanitized_Request

[everyonesmixtape] Use sanitized_Request

[extremetube] Use sanitized_Request

[facebook] Use sanitized_Request

[fc2] Use sanitized_Request

[flickr] Use sanitized_Request

[4tube] Use sanitized_Request

[gdcvault] Use sanitized_Request

[extractor/generic] Use sanitized_Request

[hearthisat] Use sanitized_Request

[hotnewhiphop] Use sanitized_Request

[hypem] Use sanitized_Request

[iprima] Use sanitized_Request

[ivi] Use sanitized_Request

[keezmovies] Use sanitized_Request

[letv] Use sanitized_Request

[lynda] Use sanitized_Request

[metacafe] Use sanitized_Request

[minhateca] Use sanitized_Request

[miomio] Use sanitized_Request

[meovideo] Use sanitized_Request

[mofosex] Use sanitized_Request

[moniker] Use sanitized_Request

[mooshare] Use sanitized_Request

[movieclips] Use sanitized_Request

[mtv] Use sanitized_Request

[myvideo] Use sanitized_Request

[neteasemusic] Use sanitized_Request

[nfb] Use sanitized_Request

[niconico] Use sanitized_Request

[noco] Use sanitized_Request

[nosvideo] Use sanitized_Request

[novamov] Use sanitized_Request

[nowness] Use sanitized_Request

[nuvid] Use sanitized_Request

[played] Use sanitized_Request

[pluralsight] Use sanitized_Request

[pornhub] Use sanitized_Request

[pornotube] Use sanitized_Request

[primesharetv] Use sanitized_Request

[promptfile] Use sanitized_Request

[qqmusic] Use sanitized_Request

[rtve] Use sanitized_Request

[safari] Use sanitized_Request

[sandia] Use sanitized_Request

[shared] Use sanitized_Request

[sharesix] Use sanitized_Request

[sina] Use sanitized_Request

[smotri] Use sanitized_Request

[sohu] Use sanitized_Request

[spankwire] Use sanitized_Request

[sportdeutschland] Use sanitized_Request

[streamcloud] Use sanitized_Request

[streamcz] Use sanitized_Request

[tapely] Use sanitized_Request

[tube8] Use sanitized_Request

[tubitv] Use sanitized_Request

[twitch] Use sanitized_Request

[twitter] Use sanitized_Request

[udemy] Use sanitized_Request

[vbox7] Use sanitized_Request

[veoh] Use sanitized_Request

[vessel] Use sanitized_Request

[vevo] Use sanitized_Request

[viddler] Use sanitized_Request

[videomega] Use sanitized_Request

[viewvster] Use sanitized_Request

[viki] Use sanitized_Request

[vk] Use sanitized_Request

[vodlocker] Use sanitized_Request

[voicerepublic] Use sanitized_Request

[wistia] Use sanitized_Request

[xfileshare] Use sanitized_Request

[xtube] Use sanitized_Request

[xvideos] Use sanitized_Request

[yandexmusic] Use sanitized_Request

[youku] Use sanitized_Request

[youporn] Use sanitized_Request

[youtube] Use sanitized_Request

[patreon] Use sanitized_Request

[extractor/common] Remove unused import

[nfb] PEP 8
ashutosh-mishra 17cc153435 Typo fix, found while going through the code.
Jaime Marquínez Ferrándiz 36e6f62cd0 Use a wrapper around xml.etree.ElementTree.fromstring in python 2.x ()
Attributes aren't unicode objects, so they couldn't be directly used in info_dict fields (for example '--write-description' doesn't work with bytes).
Sergey M․ 8cd9614abf [downloader/f4m] More accurate fragment URL construction
remitamine 324ac0a243 [downloader/f4m] get the redirected f4m_url and handle url query string properly
Yen Chi Hsuan 985e4fdc07 [downloader/hls] Add headers only for http(s) URLs
ffmpeg 2.8.1 raises an error with -headers and non-http input files.
Sergey M․ 964e7b2dd0 [downloader/common] Always skip "already downloaded" check when outputting to stdout
AndroKev f101079ae0 [downloader/rtmp] Respect --no-continue option
now when downloading a "rtmp-file" the --no-continue option works!
Sergey M․ f102819463 [downloader/hls] Pass http headers to downloader
Jaime Marquínez Ferrándiz 8a7bbd1606 [postprocessor/ffmpeg] Always use the 'file:' protocol for filenames (fixes )
If the filename contains ':' it is interpreted as a protocol.
It also handles filenames starting with '-'.
Sergey M․ 133a2b4ac2 [downloader/hls] Properly close stream
Sergey M․ 1195a38f46 [downloader/external] Use generalized cli option converters
remitamine dc534b674f [downloader/external] change _argless_option function to _valueless_option
remitamine f30c2e8e98 [downloader/external] add _argless_option for option without arguments
remitamine 266b0ad676 [downloader/external] add _bool_option to pass value to bool option
remitamine e7a8c3032d [downloader/external] Respect --no-check-certificate for curl and aria2c and --proxy for curl
Sergey M․ fcd9e423ec [hls] Properly encode fragment filename