Commit Graph

59 Commits (dbf3fa8af67dfdab42d56fdc5f35610658bc2746)

Author SHA1 Message Date
Sergey M․ a7e0531999
[downloader/http] Fix crash during urlopen caused by missing reason of URLError 4 years ago
Sergey M․ c5764b3f89
[downloader/http] Properly handle missing message in SSLError (closes #26646) 4 years ago
Sergey M․ 0837992a22
[downloader/http] Fix access to not yet opened stream in retry 4 years ago
Sergey M․ cdc55e666f
[downloader/http] Improve timeout detection when reading block of data (refs #10935) 4 years ago
Ori Avtalion 86b7c00adc
[downloader/http] Retry download when urlopen times out (#26603) (refs #10935) 4 years ago
Sergey M․ 6c907eb33f
[downloader/http] Request last data block of exact remaining size
Always request last data block of exact size remaining to download if possible not the current block size.
5 years ago
Sergey M․ f7b42518dc
[downloader/http] Finish downloading once received data length matches expected
Always do this if possible, i.e. if Content-Length or expected length is known, not only in test.
This will save unnecessary last extra loop trying to read 0 bytes.
5 years ago
Sergey M․ 3089bc748c
Fix W504 and disable W503 (closes #20863) 6 years ago
Sergey M․ 067aa17edf
Start moving to ytdl-org 6 years ago
Sergey M․ 5d6c81b63f
[downloader/http] Fix resume when writing ot stdout (closes #16699) 7 years ago
Sergey M․ b91a7a4e5e
[downloader/http] Randomize HTTP chunk size 7 years ago
Sergey M․ 00c97e3e7a
[downloader/http] Add ability to pass downloader options via info dict 7 years ago
Sergey M․ cf7259bc93
[downloader/http] Fix 302 infinite loops by not reusing requests 7 years ago
Sergey M․ ba515388b8
Introduce --http-chunk-size 7 years ago
Sergey M․ bec49996c6
[downloader/http] Return actual download result (closes #14971) 7 years ago
Sergey M․ a3c3a1e128
[http] Rework HTTP downloader
* Simplify code and split into separate routines to facilitate maintaining
* Make retry mechanism work on errors during actual download not only during connection establishment phase
* Retry on ECONNRESET and ETIMEDOUT during reading data from network
* Retry on content too short and various timeout errors
* Show error description on retry
* Closes #506, closes #809, closes #2849, closes #4240, closes #6023, closes #8625, closes #9483
7 years ago
Sergey M․ d3f0687cf7
[downloader/fragment] Use temp file for current fragment 8 years ago
Remita Amine 75a2485407 [fragment,hls,f4m,dash,ism] improve fragment downloading
- resume immediately
- no need to concatenate segments and decrypt them on every resume
- no need to save temp files for segments

and for hls downloader:
- no need to download keys for segments that already downloaded
8 years ago
Yen Chi Hsuan d54739a2e6
[downloader/http] xattr values should be bytes 8 years ago
Yen Chi Hsuan d7753d1948
[downloader/http] Use write_xattr function for --xattr-set-filesize 8 years ago
Sergey M․ 611c1dd96e [refactor] Single quotes consistency 9 years ago
Sergey M․ d800609c62 [refactor] Do not specify redundant None as second argument in dict.get() 9 years ago
Sergey M? 5c2266df4b Switch codebase to use sanitized_Request instead of
compat_urllib_request.Request

[downloader/dash] Use sanitized_Request

[downloader/http] Use sanitized_Request

[atresplayer] Use sanitized_Request

[bambuser] Use sanitized_Request

[bliptv] Use sanitized_Request

[brightcove] Use sanitized_Request

[cbs] Use sanitized_Request

[ceskatelevize] Use sanitized_Request

[collegerama] Use sanitized_Request

[extractor/common] Use sanitized_Request

[crunchyroll] Use sanitized_Request

[dailymotion] Use sanitized_Request

[dcn] Use sanitized_Request

[dramafever] Use sanitized_Request

[dumpert] Use sanitized_Request

[eitb] Use sanitized_Request

[escapist] Use sanitized_Request

[everyonesmixtape] Use sanitized_Request

[extremetube] Use sanitized_Request

[facebook] Use sanitized_Request

[fc2] Use sanitized_Request

[flickr] Use sanitized_Request

[4tube] Use sanitized_Request

[gdcvault] Use sanitized_Request

[extractor/generic] Use sanitized_Request

[hearthisat] Use sanitized_Request

[hotnewhiphop] Use sanitized_Request

[hypem] Use sanitized_Request

[iprima] Use sanitized_Request

[ivi] Use sanitized_Request

[keezmovies] Use sanitized_Request

[letv] Use sanitized_Request

[lynda] Use sanitized_Request

[metacafe] Use sanitized_Request

[minhateca] Use sanitized_Request

[miomio] Use sanitized_Request

[meovideo] Use sanitized_Request

[mofosex] Use sanitized_Request

[moniker] Use sanitized_Request

[mooshare] Use sanitized_Request

[movieclips] Use sanitized_Request

[mtv] Use sanitized_Request

[myvideo] Use sanitized_Request

[neteasemusic] Use sanitized_Request

[nfb] Use sanitized_Request

[niconico] Use sanitized_Request

[noco] Use sanitized_Request

[nosvideo] Use sanitized_Request

[novamov] Use sanitized_Request

[nowness] Use sanitized_Request

[nuvid] Use sanitized_Request

[played] Use sanitized_Request

[pluralsight] Use sanitized_Request

[pornhub] Use sanitized_Request

[pornotube] Use sanitized_Request

[primesharetv] Use sanitized_Request

[promptfile] Use sanitized_Request

[qqmusic] Use sanitized_Request

[rtve] Use sanitized_Request

[safari] Use sanitized_Request

[sandia] Use sanitized_Request

[shared] Use sanitized_Request

[sharesix] Use sanitized_Request

[sina] Use sanitized_Request

[smotri] Use sanitized_Request

[sohu] Use sanitized_Request

[spankwire] Use sanitized_Request

[sportdeutschland] Use sanitized_Request

[streamcloud] Use sanitized_Request

[streamcz] Use sanitized_Request

[tapely] Use sanitized_Request

[tube8] Use sanitized_Request

[tubitv] Use sanitized_Request

[twitch] Use sanitized_Request

[twitter] Use sanitized_Request

[udemy] Use sanitized_Request

[vbox7] Use sanitized_Request

[veoh] Use sanitized_Request

[vessel] Use sanitized_Request

[vevo] Use sanitized_Request

[viddler] Use sanitized_Request

[videomega] Use sanitized_Request

[viewvster] Use sanitized_Request

[viki] Use sanitized_Request

[vk] Use sanitized_Request

[vodlocker] Use sanitized_Request

[voicerepublic] Use sanitized_Request

[wistia] Use sanitized_Request

[xfileshare] Use sanitized_Request

[xtube] Use sanitized_Request

[xvideos] Use sanitized_Request

[yandexmusic] Use sanitized_Request

[youku] Use sanitized_Request

[youporn] Use sanitized_Request

[youtube] Use sanitized_Request

[patreon] Use sanitized_Request

[extractor/common] Remove unused import

[nfb] PEP 8
9 years ago
Sergey M․ 84bc4dcb0f [downloader/http] Clarify rationale for Content-Range check (#6426) 10 years ago
Sergey M․ 10eaa8ef1d [downloader/http] Report unable to resume 10 years ago
Sergey M․ c3124c3085 [downloader/http] Simplify 10 years ago
remitamine 8d5b8b477e [utils] import re 10 years ago
remitamine d7d2a9a3db [utils] restart download if server does not support byte ranges 10 years ago
Jaime Marquínez Ferrándiz 2ddcd88129 Remove code that was only used by the Grooveshark extractor 10 years ago
Philipp Hagemeister c59e701e35 Default to continuedl=True
We already do this in the CLI interface, so it should be just fine.
10 years ago
Jaime Marquínez Ferrándiz f28fe66970 [downloader/http] Add missing fields for _hook_progress call
It would fail if you run 'youtube-dl --no-part URL' a second time when the file has already been downloaded.

(Reported in Fedora: https://bugzilla.redhat.com/show_bug.cgi?id=1195779)
10 years ago
Jaime Marquínez Ferrándiz 123397317c [downloader/http] Remove wrong '_hook_progress' call (fixes #5117) 10 years ago
Philipp Hagemeister 09ab40b7d1 Merge branch 'progress-as-hook2' 10 years ago
Philipp Hagemeister a91a2c1a83 [downloader] Remove various unneeded assignments and imports 10 years ago
Philipp Hagemeister 16e7711e22 [downloader/http] Remove gruesome import 10 years ago
Philipp Hagemeister 5cda4eda72 [YoutubeDL] Use a progress hook for progress reporting
Instead of every downloader calling two helper functions, let our progress report be an ordinary progress hook like everyone else's.
Closes #4875.
10 years ago
Philipp Hagemeister 1283204917 [http] PEP8 (#4831) 10 years ago
light94 e77d2975af Handling Connection Reset by Peer Error 10 years ago
Philipp Hagemeister 881e6a1f5c Add --xattr-set-filesize option (Fixes #1348) 10 years ago
Jaime Marquínez Ferrándiz e1554a407d [extractors] Use http_headers for setting the User-Agent and the Referer 10 years ago
Jaime Marquínez Ferrándiz 3fcfb8e9fa [utils] YoutubeDLHandler: don't use 'Youtubedl-user-agent' for overriding the default user agent
Setting the 'User-Agent' header is enough
10 years ago
Philipp Hagemeister 1cc79574fc Fix imports and general cleanup
· Import from compat what comes from compat. Yes, some names are available in utils too, but that's an implementation detail.
· Use _match_id consistently whenever possible
· Fix some outdated tests
· Use consistent valid URL (always match the whole protocol, no ^ at start required)
· Use modern test definitions
10 years ago
Philipp Hagemeister cf372f0778 Merge remote-tracking branch 'SyxbEaEQ2/rate-limit' 10 years ago
Philipp Hagemeister 4d2d638df4 [http] Modernize 10 years ago
Philipp Hagemeister b686fc18da [hlsnative] Support test parameter 10 years ago
Sergey M․ 88a23aef5a [http] Avoid closing stdout (Fixes #3686) 10 years ago
Philipp Hagemeister d769be6c96 [grooveshark,http] Make HTTP POST downloads work 10 years ago
SyxbEaEQ2 c7667c2d7f [downloader/(common/http)] Changes calculation of the rate-limit. (Fix #2297, fix #2140, fix #595, fix #2370) 11 years ago
Sergey M․ 32fd27ec98 [http] Fix string/None comparison with int while in test 11 years ago
Sergey M․ df297c8794 [http] Improve download mechanism when Range HTTP header is ignored 11 years ago