Commit Graph

5001 Commits (226b886ca85b5843d2cb9638fb6d88223e9dd5b2)

Author SHA1 Message Date
Sergey M․ 1ddb9456c4 [extractor/generic] Use compat_urllib_parse_unquote for unquoting video_id and title from URL
Sergey M․ 58bde34a23 [extractor/generic] Force Accept-Encoding to any for extraction pass
Sergey M․ 339516072b [extractor/generic] Unescape video_id and title extracted from URL
Yen Chi Hsuan a2d971309b [porn91] Use single quotes
Yen Chi Hsuan d05a1dbe70 [porn91] Catch daily limit error
Yen Chi Hsuan a80601f8d9 [porn91] Extract more info
Yen Chi Hsuan 1c22238756 [porn91] Simplify
Yen Chi Hsuan 9ff811c5cd [porn91] PEP8
Yen Chi Hsuan 1ebc05df91 Merge branch 'porn91' of https://github.com/PeterDing/youtube-dl into PeterDing-porn91
Sergey M․ 386bdfa698 [youtube:user] Workaround 35 pages limitation (Closes )
Naglis Jonaitis 1ae7ff771b [tubitv] Add error message for videos that require login ()
Naglis Jonaitis 5196b98897 [tubitv] Add new extractor (Closes )
Sergey M․ e6e63e91a7 [tf1] Extend _VALID_URL (Closes )
Sergey M․ b4dd98358f [vgtv] Properly handle lives
Sergey M․ 4d454c5e4b [vgtv] Check for inactive videos
Sergey M․ 5c2191a605 [vgtv] Skip wasLive hds (Closes )
Sergey M․ bba5bfc890 Merge branch 'ping-soompi'
Sergey M․ 1a5b77dc21 [crunchyroll] Fix python 3.2
Sergey M․ b2cf6543b2 [soompi] Improve and simplify
Sergey M․ 0385d64223 [crunchyroll] Extract subtitles extraction routine
pulpe 6ebdfe43e4 [tube8] fix extractor (fixes )
Yen Chi Hsuan fafec39d41 [spiegeltv] Changed RTMP server (fixes and fixes )
Thanks to @brickleroux for finding out the problem
PeterDing 670861bd20 [iqiyi] Do not request for unneeded formats
PeterDing 08f7db20c1 [youku] change format_id
PeterDing 605ec701b7 [iqiyi] Add new extractor for iqiyi.com
Yen Chi Hsuan 84e1e036c2 [senate] Extend _VALID_URL (fixes )
PeterDing 1498940b10 [youku] compare bytes and str for compatible; use compat_urllib_parse for making video_url
PeterDing 806598b94d [porn91] the one that _search_regex returns not needs to be checked
Sergey M․ e26be70bca Merge branch 'soompi' of https://github.com/ping/youtube-dl into ping-soompi
PeterDing ca45246627 [youku] compatible for python > 3.3 or > 2.7
PeterDing f9355dc989 [youku] update youku
Sergey M․ 9e0b579128 [nowtv] Add test for rtlnitro
Sergey M․ ff4a1279f2 [nowtv] Do not request unnecessary metadata
Sergey M․ 9b254aa177 [nowtv] Add non-free video check
PeterDing 703d78bbf5 [porn91] change re to _search_regex
Sergey M․ d9446c7319 Merge branch 'akirk-nowtv'
Sergey M․ b25b645d51 [nowtv] Improve and simplify
PeterDing d90b3854ca [porn91] Add new extractor for 91porn.com
Sergey M․ bf24c3d017 [facebook] Improve title regex (Closes )
Yen Chi Hsuan f9f3e3df9a [teamcoco] Use determine_ext to determine the video type
Some videos does not contain a 'type' field ()
Yen Chi Hsuan f8d5e1cfb5 [naver] Fix video url (fixes )
RTMP urls in test:naver does not work. Need more investigation.
Yen Chi Hsuan c23848b3c5 [naver] Enhanced error detection
Yen Chi Hsuan 6d00a2dcd1 [bilibili] Catch API call failures
JSON are returned in a failed API call
Yen Chi Hsuan b535170b21 [bilibili] Skip assertion if HQ videos not available
Sergey M․ 1434184c57 [spankwire] Do not modify aes key string
Sergey M․ 7a372b64df [pornhub] Do not modify aes key string (Closes )
Sergey M․ 5406af92bc [dailymotion:user] Fix _VALID_URL
Sergey M․ 7d65242dc3 [dailymotion:user] Process user home as user (Closes )
Naglis Jonaitis 544a8693b7 Remove Firedrive and Sockshare imports
Oops
Naglis Jonaitis 35a4f24a37 [firedrive] Remove extractor (Closes )
Haywire since last October.
Naglis Jonaitis ff305edd64 [sockshare] Remove extractor
Haywire since last October.
Yen Chi Hsuan efec4358b9 [cinemassacre] Support an alternative form of screenwavemedia URL
fixes 
Yen Chi Hsuan db3ca36403 [facebook] Move the title extraction warning below (fixes )
Yen Chi Hsuan 42833b44b5 [tf1] Extend _VALID_URL (fixes )
Alexander Kirk 5d0a33eebc rtlnow is now hosted at nowtv.de
Sergey M․ ba2df04b41 [odnoklassniki] Make URL explicit
Sergey M․ c6bbdadd79 [odnoklassniki] Support extraction from metadata URL (Closes )
Mister Hat 680f9744c4 [pinkbike] used proper conversion methods
Mister Hat 2c935c0c72 [pinkbike] converted duration to int
Mister Hat 7198063d96 [pinkbike] new extractor
Sergey M? d41ebe146b [tenplay] Fix formats and modernize (Closes )
Jaime Marquínez Ferrándiz 4b4e1af059 [arte] Remove unused import
Sergey M. 80240b347e Merge pull request from jaimeMF/remove-nondash
[youtube] Remove the nondash formats (fixes )
Jaime Marquínez Ferrándiz 04b3b3df05 [youtube] Remove the nondash formats (fixes )
Since we use fixed values for some fields like width and height they can be wrong, and would get picked by some formats filters.
For example for https://www.youtube.com/watch?v=EQCrhbBxsjA the biggest height is 720 and for nondash formats it's set to 1440, so -f 'bestvideo[height>=1200]+bestaudio' would incorrectly pick the nondash format, instead it should report that the requested format is not available.
Sergey M․ 2ad5708c43 [arte:future] Switch to `search_regex` for now (Closes )
Sergey M․ 63f3cab4ae [rtbf] Fix extraction (Closes )
Sergey M․ d78c834ead [karrierevideos] Improve and simplify
Sergey M․ 05a976cd99 Merge branch 'karrierevideos' of https://github.com/misterhat/youtube-dl into misterhat-karrierevideos
Sergey M․ 34fb7e46ad [empflix] Relax _VALID_URL
Sergey M․ abac15f3c6 [tnaflix] Do not capture `cat_id`
Sergey M. b700055ba4 Merge pull request from frenchy1983/fix_tnaflix_regex
[TNAFlix] Allow dot (and more) in cat_id and display_id
Sergey M? 1335c3aca8 [drtv] Improve extraction (Closes )
Yen Chi Hsuan 30455ce255 [nextmedia] Extend and reorder _VALID_URL
Yen Chi Hsuan 9bf87ae3aa [nextmedia] Merge AppleDailyRealtimeNewsIE and AppleDailyAnimationNewsIE
Yen Chi Hsuan abca34cbc0 [cnn] Relax _VALID_URL again (fixes )
The problem is the same as test:CNN_1, so I didn't add the test case
Sergey M․ d386878af9 [prosiebensat1] Add support for .at domain names (Closes )
Sergey M․ 685c74d315 [rutv] Extend embed URL (Closes )
Jaime Marquínez Ferrándiz ba64547616 [sportbox] Remove unused import
frenchy1983 ed5a637d62 [TNAFlix] Restore test
See dstftw's comment in 
Yen Chi Hsuan 8a278a1d7e [nba] Fix duration extraction (fixes )
Sergey M․ 77d9cb2f04 [sportbox] Fix extraction
Sergey M․ 0459432d96 [shared] Fix for python 3.2
Sergey M․ 43150d7ac3 [shared] Fix for python 3.2
Sergey M․ afe8b594be [rtve.es:alacarta] Fix for python 3.2
Sergey M․ 06947add03 [chilloutzone] Fix for python 3.2
Sergey M․ 5cd47a5e4f [videott] Fix for python 3.2
Sergey M․ 53de95da5e [viki] Extend _VALID_URLs
Jaime Marquínez Ferrándiz 6ad9cb224a [mitele] It now uses m3u8 ()
It should also be possible to use Adobe HDS, but it would require more work.
frenchy1983 e7752cd578 [TNAFlix] Allow dot (and more) in cat_id and display_id
URLs with dots were raising a "UnsupportedError: Unsupported URL" error.
Jaime Marquínez Ferrándiz 4d2f42361e [viki] remove unused import
Sergey M․ 4d8ee01389 [viki] Fix typo
Sergey M․ d01924f488 [viki:channel] Extend matching URLs and extract movies
Sergey M․ bc56355ec6 [viki:channel] Switch to API
Sergey M․ ac20d95f97 [viki] Add support for youtube externals
Sergey M․ 1a83c731bd [viki] Switch extraction to API
Sergey M․ ca57a59883 Merge branch 'ping-viki-shows'
Sergey M․ b0d619fde2 [viki:channel] Extract title from JSON
Sergey M․ cc7051efd7 Merge branch 'viki-shows' of https://github.com/ping/youtube-dl into ping-viki-shows
ping 5137adb94d [soompi] Switch to non-geoblocked test video
ping 2632941f32 [soompi] Add new extractor for tv.soompi.com
ping 137597b0ea [dramafever] Streamline code
Yen Chi Hsuan 051df9ad99 [letv/sohu] Skip tests relying on external proxies
The proxy is currently broken. See  and 
ping f670ef1c8e [dramafever] Add new extractor for dramafever.com
Sergey M․ d9d747a06a [ultimedia] Fix extraction
Yen Chi Hsuan b813d8caf1 [qqmusic] Unescape '\\n' in description ()
Yen Chi Hsuan ecee572411 [yahoo] Add support for closed captions (closes )
Sergey M․ 484c9d2d5b [vier] Fix extraction
Sergey M․ 5d8dcb5342 [vuclip] Fix extraction
Sergey M․ 2328f2fe68 [vulture] Fix extraction
Sergey M․ 4f514c7e88 [wimp] Fix youtube extraction (Closes )
Sergey M․ 5bdc520cf1 [xminus] Fix extraction
Jaime Marquínez Ferrándiz fc6e75dd57 [instagram] Only recognize https urls (fixes )
http urls redirect to them.
Mister Hat ba9d16291b manually specify namespace
Mister Hat 725652e924 [karrierevideos] add support for www.karrierevideos.at (closes )
ping 8da0e0e946 [viki] Change IE name to channel, better message output
Sergey M․ 588b82bbf8 [tv2:article] Add extractor (Closes )
Sergey M․ bc0f937b55 [tv2] Add extractor ()
Sergey M․ baa43cbaf0 [extractor/common] Relax valid url check verbosity
Sergey M․ adb6b1b316 Merge branch 'viki-shows' of https://github.com/ping/youtube-dl into ping-viki-shows
ping 1c18de0019 [viki] Add proper paging and include clips
Jaime Marquínez Ferrándiz 4d52f2eb7f [sbs] Remove unused import
Sergey M․ 363cf58645 Merge branch 'viki-shows' of https://github.com/ping/youtube-dl into ping-viki-shows
Sergey M․ 7e760fc188 [espn] Add extractor ()
Unfinished
Sergey M․ ef2dcbe4ad [sbs] Fix extraction (Closes )
Sergey M․ 9354a5fad4 [ooyala] Fix unresolved reference
Sergey M․ 1c97b0a777 [ooyala:external] Add extractor
ping 2f3bdab2b9 [viki] Fix code format
ping 0d7f036429 [viki] Add support for shows
Sergey M. 2cda13213d Merge pull request from blissland/master
[CBSNewsIE] Relax thumbnail regex so test passes
Sergey M․ 70d0d43b5e [rts] Check formats (Closes )
Sergey M․ 25c3a7348f [generic] Fix typo
Sergey M․ 9123d64592 Merge branch 'maddoger-sportbox-fix'
Sergey M․ b827a6015c [generic] Add test for sportbox embeds
Sergey M․ d40a3b5b55 [generic] Add support for sportbox embeds
Sergey M․ ef28a6cb26 [sportbox:embed] Relax thumbnail
Sergey M․ 1436a6835e [sportbox:embed] Add `_extract_urls`
blissland e8cfacae37 [CBSNewsIE] Relax thumbnail regex so test passes
Sergey M․ 3a7382950b [sportbox:embed] Add extractor
Jaime Marquínez Ferrándiz eeb23eb7ea [gamespot] The protocol is not optional
Jaime Marquínez Ferrándiz 34fe5a94ba [gamespot] Add support for videos that don't use 'f4m_stream' (fixes )
Sergey M․ 6181864290 Merge branch 'sportbox-fix' of https://github.com/maddoger/youtube-dl into maddoger-sportbox-fix
Vitaliy Syrchikov e9ca615a98 New test
Sergey M․ 62c95fd5fc [youtube:feed] Check each 'load more' portion for unique video ids
Sergey M․ 25f14e9f93 [youtube] Separate feed extractor
Vitaliy Syrchikov ae670a6ed8 Sportbox source fix. HD videos support.
Vitaliy Syrchikov a7b8467ac0 Sportbox extractor fix.
blissland 15da7ce7fb Fix file format extraction regex and update test file checksum
Jaime Marquínez Ferrándiz 2bc4330303 [youtube:history] Fix extraction (fixes )
It uses the same method as YoutubeSubscriptionsIE, if other feed starts using it we should consider using base class.
Yen Chi Hsuan 12675275a1 [teamcoco] Detect expired videos ()
Yen Chi Hsuan 3a105f7b20 [teamcoco] Rewrite preload data extraction
Idea: "puncture" some consecutive fragments and check whether the
b64decode result of a punctured string is a valid JSON or not.

It's a O(N^3) algorithm, but should be fast for a small N (less than 30
fragments in all test cases)
Sergey M․ 1ae72fb23d [soundcloud:user] Defer download link resolve (Closes )
Looks like final download links can expire before downloading process reach them. So, resolving download links right before actual downloading.
Yen Chi Hsuan 7ec676bb3d [qqmusic] Add IE_NAME for all extractors
Yen Chi Hsuan 29ea57283e [qqmusic] Refactoring QQMusicToplistIE
Yen Chi Hsuan 5488973961 [qqmusic] flake8
Yen Chi Hsuan 96d45a5489 Merge pull request from ping/qqmusic-toplist-ie
[qqmusic] Add support for charts / top lists
Sergey M․ 7a012d5a16 [screenwavemedia] Add support for player2 URLs (Closes )
Yen Chi Hsuan fa6a16996e [worldstarhiphop] Support Android URLs (fixes )
Sergey M․ 7d57d2e18b [canalplus] Restore checksums in tests
blissland 509c630db8 [CanalplusIE] Update tests that were no longer working
Sergey M․ 372744c544 [odnoklassniki] Fix extraction (Closes )
blissland 0b4253fa37 [BYUtvIE] Change thumbnail regex so test does not fail
ping 86ec1e487c [qqmusic] Code fixes
ping fd4eefed39 [qqmusic] Fix extraction for global list
ping b480e7874b [qqmusic] Fix code formatting
ping 41333b97b9 [qqmusic] Add support for charts / top lists
Yen Chi Hsuan c1c924abfe [utils,common] Merge format_srt_time and _subtitles_timecode
format_srt_time uses a comma as the delimiter between seconds and
milliseconds while _subtitles_timecode uses a dot. All .srt examples I
found on the Internet uses a comma, so I use a comma in the merged
version. See http://matroska.org/technical/specs/subtitles/srt.html and
http://devel.aegisub.org/wiki/SubtitleFormats/SRT
Yen Chi Hsuan 1c7e2e64f6 [nrk] Remove TTML to srt conversion codes
A common routine is implemented in utils.py and can be used via
--convert-subtitles.
Yen Chi Hsuan 5332fd91bf [nytimes] Correct _VALID_URL of NYTimesArticleIE
Sergey M․ d4b963d0a6 [vine] Relax `alt_title` (Closes )
Sergey M․ 6d3f5935e5 [southpark] Fix IE_NAME
rrooij 968ee17677 [southparkdk] Add extractor
rrooij 81ed3bb9c0 [southpark] Sort alphabetically
Sergey M․ 5115652828 [zingmp3] Capture error message
Sergey M․ 1f92865494 [dumpert] Add cpc cookie (Closes )
Yen Chi Hsuan e41f450f28 [tmz] Add support for articles (fixes )
Sergey M․ 370b39e8ec [voicerepublic] Fix fallback branch formats extraction
Sergey M․ a6762c4a22 [voicerepublic] Make more robust and extract more metadata
Sergey M․ 98c2c0febc Merge branch 'voicerepublic' of https://github.com/duncankl/youtube-dl into duncankl-voicerepublic
Yen Chi Hsuan 63cbd19f50 [ndr] Replace the 404 test case
Yen Chi Hsuan 1934f3a0ea [ndr] Extended to support n-joy.de as well (closes )
According to http://en.wikipedia.org/wiki/N-Joy, n-joy.de is a service
hosted by NDR, so I put them together.
ping a909e6ad43 [dailymotion] Patch upload_date detection.
(closes )
Duncan 1dcb52188d [voicerepublic] Remove hardcoded paths to media files
Duncan 28ebef0b1b [voicerepublic] Detect list of available formats from the web page
Duncan f03a8a3c4e [voicerepublic] Raise ExtractorError if audio is still being processed
Duncan 03f760b1c0 [voicerepublic] Remove creator field
Duncan f900dc3fb9 [voicerepublic] Extract author using _html_search_meta
Sergey M․ 95eb1adda8 [life:embed] Sort formats
Duncan c6ddbdb66c [voicerepublic] Add new extractor
Sergey M․ 3800b908b1 [mlb] Fix
Sergey M․ 754270313a [life:embed] Move to separated extractor and extract m3u8 formats
Sergey M․ 057ebeaca3 [lifenews] Add test for
Sergey M․ 480065172d [lifenews] Add support for video URLs (Closes )
Sergey M․ f2e0056579 [vgtv] Avoid duplicate format_id
Sergey M․ 32fffff2cc [eroprofile] Fix video URL extraction (Closes )
blissland 0892090a56 Added audio test for BRIE
blissland d592b42f5c Updated two tests for BRIE
Jaime Marquínez Ferrándiz 3b5f65a64c [mlb] Fix extraction of articles
And move test from generic, since it's directly handled by MLBIE
Jaime Marquínez Ferrándiz 5c0b2c16a8 [vgtv] Escape '#' in _VALID_URL and remove empty newlines at the end
In verbose mode, '#' is interpreted as the start of a comment.
Yen Chi Hsuan 6d14d08e06 [yam] Fix title and uploader id
Yen Chi Hsuan 32060c6d6b [sohu] Update extractor
The original extraction logic always fails for all test videos
Yen Chi Hsuan 3dbec410a0 [sohu] Enhance error handling
Sergey M․ de765f6c31 [foxsports] Support some more URLs ()
Sergey M․ dc455a5f88 [extractor/generic] Add test for svt embed
Sergey M․ bab19a8e91 [extractor/generic] Add support for svt embeds (Closes )
Sergey M․ 322915014f [svtplay] Rename to svt
Sergey M․ 79998cd5af [svtplay] Generalize svt extractors and add svt.se extractor
Sergey M․ 4384cf9e7d [extractor/__init__] Fix alphabetic order
Sergey M․ fe373287eb [vgtv] Add support for bt vestlendingen (Closes )
Sergey M․ cbe443362f [aftenposten] Implement in terms of xtream extractor
Sergey M․ 2c0c9dc46c [xstream] Move xstream to separate extractor
Sergey M․ 0ceab84749 [vgtv] Add support for bt.no articles ()
Sergey M․ 34e7dc81a9 [vgtv] Add support for generic bt.no URLs ()
Sergey M․ 4e6e9d21bd [mlb] Improve _VALID_URL
Sergey M․ d1feb30811 [mlb] Fallback to extracting video id from webpage for all URLs that does not contain it explicitly (Closes )
blissland 43837189c1 Fix URL template extraction for netzkino. Fixes
blissland 249962ffa2 [bet] Use unique part of xml url as the video id and fix tests (closes )
The guid changes often.
Yen Chi Hsuan 7ef00afe9d [nhl] Support RTMP videos (fixes )
Naglis Jonaitis 46be82b811 [vessel] Use `main_video_asset` when searching for video_asset (Fixes )
Yen Chi Hsuan 09b412dafa [nhl] Partial support for hlg id (fixes )
Jaime Marquínez Ferrándiz 5268a05e47 [ooyala] Style fix
Sergey M․ 406224be52 [extractor/generic] Fix following incomplete redirects ()
Yen Chi Hsuan 553e412bda Merge branch 'master' of github.com:rg3/youtube-dl
Sergey M․ f22834a372 [bild] Relax thumbnail test check
blissland bc08873cff Fix indents
Yen Chi Hsuan aafe273990 [ooyala] Use SAS API to extract info (fixes )
blissland c09593c04e [BildIE] Escape ampersands in xml and update test thumbnail
Yen Chi Hsuan 84bf31aaf8 [ooyala] Extract m3u8 information ()
Yen Chi Hsuan 05d5392cda [common] Ignore subtitles in m3u8
Yen Chi Hsuan d9a743d917 [vice] Remove a redundant print
Yen Chi Hsuan ac6c358c2a [teamcoco] Fix extracting preload data again
Sergey M․ ad0c0ad3b4 [historicfilms] Fix tape id extraction
Sergey M․ 1ed34f3dd6 [gorillavid] Switch 404 test to only matching
Sergey M․ 6a8f9cd22e [giga] Fix view count extraction
Sergey M․ e8b9ab8957 [pbs] Add `format_id` for direct links
Sergey M․ 74f728249f [extractor/common] Fallback to empty string for (yet) missing `format_id` in `_sort_formats` (Closes )
blissland d6a1738892 [archive.org] Fix incorrect url condition (closes )
The condition for assigning to json_url is the wrong way round:

currently for url: aaa.com/xxx

we get:

aaa.com/xxx&output=json

instead of the correct value:

aaa.com/xxx?output=json
Sergey M․ b326b07adc [lifenews] Use `_proto_relative_url`
Yen Chi Hsuan 07d2921c6d [lifenews] Correctly determine iframe links (fixes )
rrooij dcf8077906 [southparknl] Fix test to match playlist tests
rrooij 3408f6e64a [southparkde] Fix naming inconsistency
The class was first called 'SouthparkDe'. It is now changed to
'SouthParkDe' to match the name of the other extractors.
rrooij e10dc0e1f0 [southparknl] Add extractor for southpark.nl
Sergey M․ ce5c1ae517 [noco] Remove unused import
Sergey M․ 01e4b1ee14 [noco] Update tests
Sergey M․ 815ac0293e [noco] Modernize
Sergey M․ 6568382d6f [noco] Extract all variations of audio/subtitles media
Sergey M․ f943b7ddce Merge branch 'noco' of https://github.com/Tassatux/youtube-dl into Tassatux-noco
Aurélien Dunand ff9d68e7be [noco] Add test for multi languages video
Aurélien Dunand 7212560f4d [noco] Retrieve video language according to user options
Sergey M․ 1aa43d77c0 [rutv] Remove superfluous check
Sergey M․ e038d5c4e3 [rutv] Fix preference