c047270c02
[utils] Remove Content-encoding from headers after decompression
...
With cn_verification_proxy, our http_response() is called twice, one from
PerRequestProxyHandler.proxy_open() and another from normal
YoutubeDL.urlopen(). As a result, for proxies honoring Accept-Encoding, the
following bug occurs:
$ youtube-dl -vs --cn-verification-proxy https://secure.uku.im:993 "test:letv"
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['-vs', '--cn-verification-proxy', 'https://secure.uku.im:993 ', 'test:letv']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2015.12.23
[debug] Git HEAD: 97f18fahttp://www.letv.com/ptv/vplay/22005890.html
[Letv] 22005890: Downloading webpage
[Letv] 22005890: Downloading playJson data
ERROR: Unable to download JSON metadata: Not a gzipped file (b'{"') (caused by OSError('Not a gzipped file (b\'{"\')',)); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see https://yt-dl.org/update on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
File "/home/yen/Executables/Multimedia/youtube-dl/youtube_dl/extractor/common.py", line 330, in _request_webpage
return self._downloader.urlopen(url_or_request)
File "/home/yen/Executables/Multimedia/youtube-dl/youtube_dl/YoutubeDL.py", line 1886, in urlopen
return self._opener.open(req, timeout=self._socket_timeout)
File "/usr/lib/python3.5/urllib/request.py", line 471, in open
response = meth(req, response)
File "/home/yen/Executables/Multimedia/youtube-dl/youtube_dl/utils.py", line 773, in http_response
raise original_ioerror
File "/home/yen/Executables/Multimedia/youtube-dl/youtube_dl/utils.py", line 761, in http_response
uncompressed = io.BytesIO(gz.read())
File "/usr/lib/python3.5/gzip.py", line 274, in read
return self._buffer.read(size)
File "/usr/lib/python3.5/gzip.py", line 461, in read
if not self._read_gzip_header():
File "/usr/lib/python3.5/gzip.py", line 409, in _read_gzip_header
raise OSError('Not a gzipped file (%r)' % magic)
9 years ago
9b9c5355e4
Rename error_to_str to error_to_compat_str
9 years ago
8e60dc7526
[utils] Add encode_compat_str
9 years ago
fdae235858
[utils] Add error_to_str
9 years ago
db2fe38b55
[utils] Support alternative timestamp format in TTML
...
Fixes #7608
9 years ago
d631d5f9f2
[utils] Fix TTML conversion
...
Tolerate invalid timestamps (closes #7909 )
9 years ago
31b2051e21
[utils] Add remove_quotes
9 years ago
992fc9d6e1
[utils] Refactor handle_youtubedl_headers for future extension
9 years ago
0424ec307b
[utils] Correct docstring of YoutubeDLHandler
9 years ago
87f0e62d94
[utils] Separate codes for handling Youtubedl-* headers
9 years ago
67dda51722
Rename compat_urllib_request_Request to sanitized_Request and move to utils
9 years ago
9cb9a5df77
[utils] Check ext with trailing slash against the list of known extensions
9 years ago
3e12bc583a
[utils] Improve determine_ext ( Closes #7593 )
9 years ago
7e1f5447e7
[utils] Improve encode_dict
9 years ago
7a3f0c00ad
[utils] Style
9 years ago
7aefc49c40
[utils] Skip invalid/non HTML entities ( Closes #7518 )
9 years ago
6a75040278
[utils] unified_strdate: Return None if the date format can't be recognized ( fixes #7340 )
...
This issue was introduced with ae12bc3ebb
9 years ago
c90d16cf36
[utils:sanitize_path] Disallow trailing whitespace in path segment ( Closes #7332 )
9 years ago
30eecc6a04
Merge pull request #7296 from jaimeMF/xml_attrib_unicode
...
Use a wrapper around xml.etree.ElementTree.fromstring in python 2.x (…
9 years ago
ae12bc3ebb
[utils] Make unified_strdate always return unicode string
9 years ago
578c074575
[utils] Support list of xpath in xpath_element
9 years ago
52c3a6e49d
[utils] Improve parse_iso8601
9 years ago
f78546272c
[compat] compat_etree_fromstring: also decode the text attribute
...
Deletes parse_xml from utils, because it also does it.
9 years ago
36e6f62cd0
Use a wrapper around xml.etree.ElementTree.fromstring in python 2.x ( #7178 )
...
Attributes aren't unicode objects, so they couldn't be directly used in info_dict fields (for example '--write-description' doesn't work with bytes).
9 years ago
d01949dc89
[utils:js_to_json] Fix bad escape in double quoted strings
9 years ago
1e399778ee
[letv] Fix extraction
...
Using data URIs for passing the decrypted M3U8 manifest, which is
supported by ffmpeg only.
9 years ago
af98f8ff37
[utils] Return default on fail in int_or_none
9 years ago
caf80631f0
[utils] Do not fail in float_or_none on non-numeric data
9 years ago
1812afb7b3
[utils] Do not fail in int_or_none on non-numeric data ( Closes #7175 )
9 years ago
5a1a2e9454
[utils] Fix kwargs on old python 2 ( Closes #6905 )
9 years ago
e28034c5ac
[utils] Comment cookie processing until result from travis and some more testing
9 years ago
266e466ee4
[utils] Simplify cookie processor
9 years ago
1639282434
[utils] Add encode_dict
9 years ago
ad72917274
[utils] Add issue URL in comment for #6457
9 years ago
a6420bf50c
[utils] Add cookie processor for cookie correction ( Closes #6769 )
9 years ago
66e289bab4
[utils] Generalize cli option converters
9 years ago
8e636da499
[utils] Improve xpath_text
9 years ago
5d2354f177
[utils] Relax attribute key assert
9 years ago
a41fb80ce1
[utils] Add xpath_element and xpath_attr
9 years ago
e5e78797e6
[utils] Strict HTTP responses ( Closes #6727 )
9 years ago
5a4d9ddb21
[utils] Percent-encode redirect URL of Location header ( Closes #6457 )
9 years ago
51f267d9d4
[YoutubeDL:utils] Move percent encode non-ASCII URLs workaround to http_request and simplify ( Closes #6457 )
9 years ago
ee114368ad
[utils] Make value optional for find_xpath_attr
...
This allows selecting particular attributes by name but without specifying the value and similar to xpath syntax `[@attrib]`
9 years ago
2c7ed24796
Remove redundant (and wrong) class parameters
9 years ago
9c29bc69f7
[utils] Improve parse_duration
...
Now dots are parsed. For example '87 Min.'
9 years ago
bf42a9906d
[utils] Add default value for xpath_text
10 years ago
4eb10f6621
[utils] Add ISO3166Utils
10 years ago
4e33577173
[utils] Support ttaf1 namespace in TTML
...
It's found in bbc.co.uk. See #6038
10 years ago
396726244a
[utils/ffmpeg] Move ISO 639 related codes to utils
10 years ago
ecee572411
[yahoo] Add support for closed captions ( closes #5714 )
10 years ago
1b0427e6c4
[utils] Support TTML without default namespace
...
In a strict sense such TTML is invalid, but Yahoo uses it.
10 years ago
c1c924abfe
[utils,common] Merge format_srt_time and _subtitles_timecode
...
format_srt_time uses a comma as the delimiter between seconds and
milliseconds while _subtitles_timecode uses a dot. All .srt examples I
found on the Internet uses a comma, so I use a comma in the merged
version. See http://matroska.org/technical/specs/subtitles/srt.html and
http://devel.aegisub.org/wiki/SubtitleFormats/SRT
10 years ago
7dff03636a
[utils] Support 'dur' field in TTML
10 years ago
d39e0f05db
[utils] Remove sanitize_url_path_consecutive_slashes()
...
This function is used only in SohuIE, which is updated to use a new
extraction logic.
10 years ago
541168039d
[utils] get_exe_version: encode executable name ( fixes #5647 )
...
It failed in python 2.x when $PATH contains a directory with non-ascii characters.
10 years ago
0fe2ff78e6
[NBC] Enhance embedURL extraction ( closes #2549 )
10 years ago
b3ed15b760
[utils] Add replace_extension
10 years ago
e65e4c8874
[utils] Improve prepend_extension
...
Now `ext` is appended to filename if real extension != expected extension.
10 years ago
c86b61428b
[utils] Fix another old python 2.6 kwargs issue ( Closes #5539 )
10 years ago
40b96352c9
Merge pull request #5523 from jaimeMF/remove-format-limit
...
Remove the --max-quality option
10 years ago
aa49acd15a
[utils] Add `get_subprocess_encoding` and filename/argument decode counterparts
10 years ago
bf6427d2fb
[ffmpeg] Add dfxp (TTML) subtitles support ( #3432 , #5146 )
10 years ago
9f3fa89f7c
Remove the --max-quality option
...
It doesn't work well with 'bestvideo' and 'bestaudio' because they are usually before the max quality.
Format filters should be used instead, they are more flexible and don't require the requested quality to exist for each video.
10 years ago
08f2a92c9c
InfoExtractor._search_regex: Suggest updating when the regex is not found (suggested in #5442 )
...
Reuse the same message from ExtractorError
10 years ago
be531ef1ec
[utils] Fix splitunc deprecation warning
10 years ago
0a1603634b
[utils] Remove url_infer_protocol
10 years ago
418c5cc3fc
[udn] Add new extractor
10 years ago
15ac8413c7
[utils] Avoid treating `*-%Y` date template as UTC offset
10 years ago
79c21abba7
[utils] Add one more template to unified_strdate
10 years ago
ba9e68f402
[utils] Drop trailing comma before closing brace
10 years ago
70a1165b32
Don't use bare 'except:'
...
They catch any exception, including KeyboardInterrupt, we don't want to catch it.
10 years ago
91757b0f37
[utils] Escape all HTML entities written in hexadecimal form
10 years ago
92a4793b3c
[utils] Place sanitize url function near other sanitizing functions
10 years ago
dc03a42537
Merge branch 'sohu_fix' of https://github.com/yan12125/youtube-dl into yan12125-sohu_fix
10 years ago
2ebfeacabc
[utils] Keep dot and dotdot unmodified ( Closes #5171 )
10 years ago
f18ef2d144
[utils] Disallow trailing dot in sanitize_path for a path part
10 years ago
d55de57b67
[utils] Fix sanitize_open
10 years ago
a2aaf4dbc6
[utils] Add sanitize_path
10 years ago
55969016e9
[utils] Add a function to sanitize consecutive slashes in URLs
10 years ago
2461f79d2a
[utils] Correct per-request proxy handling
10 years ago
91410c9bfa
[letv] Add --cn-verification-proxy ( Closes #5077 )
10 years ago
a7440261c5
[utils] Streap leading dots
...
Fixes #2865 , closes #5087
10 years ago
003c69a84b
Use shutil.get_terminal_size for getting the terminal width if it's available (python >= 3.3)
10 years ago
3e675fabe0
[airmozilla] Be more tolerant when nonessential items are missing ( #5030 )
10 years ago
183139340b
[utils] Bump our user agent
10 years ago
5a42414b9c
[utils] Prevent hyphen at beginning of filename ( Fixes #5035 )
10 years ago
8fb3ac3649
PEP8: W503
10 years ago
c460bdd56b
[sandia] Add new extractor ( #4974 )
10 years ago
d305dd73a3
[utils] Fix js_to_json
...
Previously, the runtime could be atrocious for longer inputs.
10 years ago
7105440cec
[Yam] Add new extractor
10 years ago
08b38d5401
[camdemy] Simplify and make more robust ( #4938 )
...
Do not throw errors if view count or upload date extraction fails.
Dispose of re.MULTILINE, which had absolutely no effect without any ^ or $ in sight.
Follow PEP8 naming conventions.
10 years ago
347de4931c
[YoutubeDL] Add generic video filtering ( Fixes #4916 )
...
This functionality is intended to eventually encompass the current format filtering.
10 years ago
9bb8e0a3f9
[wsj] Add new extractor ( Fixes #4854 )
10 years ago
8f4b58d70e
[ntvde] Add new extractor ( Fixes #4850 )
10 years ago
8f9312c387
Appease pyflakes8-3
10 years ago
4f264c02c7
[utils] YoutubeDLHTTPSHandler.https_open: pass all required arguments to do_open
...
With this change the '--no-check-certificate' works again (#4807 ).
10 years ago
cfb56d1af3
Add --list-thumbnails
10 years ago
3fcfb8e9fa
[utils] YoutubeDLHandler: don't use 'Youtubedl-user-agent' for overriding the default user agent
...
Setting the 'User-Agent' header is enough
10 years ago
a055469faf
[downloader] Improve downloader selection
10 years ago
aa42e87340
[utils] Catch strange Windows errors ( Closes #4733 )
10 years ago
Yen Chi Hsuan
Sergey M․
Jaime Marquínez Ferrándiz
Raphael Michel
Naglis Jonaitis
Philipp Hagemeister