Commit Graph

7352 Commits (cf9cf7dd04d8e15635084dcf9f7a76acff5ac218)

Author SHA1 Message Date
Sergey M․ 360075e28a [pornhub] Extend _VALID_URL (Closes )
Sergey M․ accf79b107 [viki] Add support for authentication (Closes )
ping 4d58b24c15 [qqmusic] Use _check_formats instead
Yen Chi Hsuan c33a8639a7 [postprocessor/embedthumbnail] Skipping if thumbnail file missing
Fixes 
Yen Chi Hsuan 25fa8d66e6 [bbccouk] Fix description (closes )
Yen Chi Hsuan 974a6146fe [bbccouk] Fix error handling
ping 0392ac98d2 [qqmusic] Fix code formatting
ping 5e3915cbe3 [qqmusic] Fix song extraction when certain formats are unavailable
ping 29b809de68 [qqmusic] Fix album extraction
ping 0d0d5d3717 [qqmusic] Add support for playlists
Sergey M․ a69e8bfdd9 [lifenews] Fix tests
Sergey M․ 062a3fdf36 [lifenews] Modernize
Sergey M․ 028a33d7f2 [lifenews] Fix extraction
Sergey M․ 447053668f [spankwire] Fix extraction
Jaime Marquínez Ferrándiz f3aecb27a4 [youku] Simplify a bit the 'entries' construction
Mainly avoid having to use an index.
Yen Chi Hsuan 79fe954d79 Merge branch 'PeterDing-youku'
Yen Chi Hsuan 0501bfa159 [YoutubeDL] Youku extractor now uses the standard format selection
Yen Chi Hsuan a155b7e76c [youku] Coding style
Yen Chi Hsuan 5228b756af [youku] Add cn_verification_proxy support and add a georestricted test case
Yen Chi Hsuan 04e7596680 [youku] Better error handling
Yen Chi Hsuan f1e66cb2eb [youku] Change video_id and add a multipart test case
Sergey M․ 4fd35ee072 [safari] Add test for
Yen Chi Hsuan ee69799262 [youku] Add a v.swf test case
Sergey M. 636a9637f4 Merge pull request from zx8/master
[safari] make url regex more lenient
Yen Chi Hsuan 9383e66f94 [youku] Use _match_id
Sergey M․ 99e6833c85 [francetv] Update f4m manifest token URL (Closes , Closes )
Yen Chi Hsuan c203be3fb4 [youku] Better handling for Python 2/3 compatibility
Sergey M․ 02175a7986 [youtube:search] Fix search query (Closes )
Sergey M․ 8117df4cd9 [vk] Extract view count
Sergey M․ 7c7dd9dc7f [vk] Fix upload date extraction
Sergey M․ 054932f403 [vk] Fix extraction (Closes )
Yen Chi Hsuan aed473ccf9 [youku] PEP8
Yen Chi Hsuan 8268e94cd4 Merge branch 'youku' of https://github.com/PeterDing/youtube-dl into PeterDing-youku
Yen Chi Hsuan 4af98ecdfb [vbox7] Fix extraction (fixes )
Philipp Hagemeister 4b9f9010b0 release 2015.06.15
zx8 2a0fcf6113 [safari] make url regex more lenient
Sergey M․ 67d95f177c [niconico] Simplify format info
Sergey M. 44773ad125 Merge pull request from chaoskagami/niconico_qualitynote
Quality note for niconico
Sergey M․ 5774ef35c4 [options] Add missing whitespace for --fixup description
Sergey M․ b95cfa9170 [liveleak] Clarify test
Sergey M․ afa1ded425 [liveleak] Clarify rationale for restoring raw video
Sergey M․ 00ac23e6e0 [liveleak] Improve regex for restoring original video URL
jomo 8f75761f24 LiveLak: add test for URLs with 'h264_270p'
jomo 9fd24e3a22 LiveLeak: support more original videos
some (old?) videos use ...mp4.h264_270p.mp4... instead of ...mp4.h264_base.mp4...
This is an addition to 
Naglis Jonaitis 755a9d3d1a [tvplay] Add support for NovaTv
Shrimadhav U K 180940e02d spelling mistake corrected
acces changed to accessing
chaoskagami 976b03c56b Quality note for niconico - at least notify whether you'll get low or src
Sergey M․ 450d89ddc1 [dramafever] Improve _VALID_URL
Sergey M․ 463b2e5542 [dramafever:series] Rollback _PAGE_SIZE to max possible
Sergey M․ 70a2002399 [dramafever:series] Fix _VALID_URL (Closes )
Sergey M․ 0029071adb [dramefever] Improve and simplify
Sergey M․ ad49fe7c8f Merge branch 'dramafever' of https://github.com/ping/youtube-dl into ping-dramafever
Sergey M․ af9cdee9cb [brightcove] Improve and generalize brightcove URL extraction from JS
fnord b4e1576aee Brightcove extractor: support customBC.createVideo(...); method
found in http://www.americanbar.org/groups/family_law.html and
http://america.aljazeera.com/watch/shows/america-tonight/2015/6/exclusive-hunting-isil-with-the-pkk.html
Sergey M․ 78e2b74bb9 [tumblr] Add support for pornhub embeds (Closes )
Sergey M․ 65d161c480 [extractor/generic] Add support for pornhub embeds
Sergey M․ 9fcbd5db2a [pornhub] Add support for embeds
Sergey M․ 4f3bf679f5 [vk] Fix authentication for non-ASCII login/password
Sergey M․ 0072afca8e [YoutubeDL] Remove force_generic_extractor arg from process_ie_result
Sergey M․ 61aa5ba36e [YoutubeDL] Remove global state for force_generic_extractor flag in favor of passing argument
Sergey M․ 9f4323252a [YoutubeDL] Fix for multiple URLs
Sergey M․ 8b6c896c4b [prosiebensat1] Add title regex
Sergey M․ 185dbc4974 [prosiebensat1] Fix rtmp extraction (Closes )
Sergey M․ 3d535e0471 [tvc] Fix embed regex
Sergey M․ d22dec74ff Add `--force-generic-extractor`
For some extractors that are hard to workout a good _VALID_URL we use very vague and unrestrictive ones,
e.g. just allowing anything after hostname and capturing part of URL as id.
If some of these extractors happen to have an video embed of some different hoster or platform
and this scenario was not handled in extractor itself we end up with inability to download this embed
until extractor is fixed to support embed of this kind.
Forcing downloader to use the generic extractor can be a neat temporary solution for this problem.
Example: FiveTV extractor with Tvigle embed - http://www.5-tv.ru/rabota/broadcasts/48/
Sergey M․ 9872d3110c [extractor/generic] Add support for tvigle embeds
Sergey M․ b859971873 [extractor/generic] Rename tvc embed url variable
Sergey M․ 499a077761 [5tv] Improve
Sergey M․ 5da7177729 Merge branch '5tv' of https://github.com/hlintala/youtube-dl into hlintala-5tv
Sergey M․ f37bdbe537 [extractor/generic] Add test for tvc embed
Sergey M․ 2da09ff8b0 [extractor/generic] Fix tvc ie_key
Sergey M․ 5ccddb7ecf [tvc] Fix ie_key
Sergey M․ 954c1d0529 [tvc] Refactor extractor names
Sergey M․ 494f20cbdc [extractor/generic] Add support for tvc embeds
Sergey M․ 29902c8ec0 [tvc:embed] Add embed extraction routine
Sergey M․ 9f15bdabc8 [tvc] Separate embed extractor
Sergey M․ fff3455f58 Merge branch 'tvc' of https://github.com/hlintala/youtube-dl into hlintala-tvc
Hannu Lintala 87446dc618 [tvc] Add extractor (Closes )
Hannu Lintala 99ac0390f5 [fivetv] Add extractor (Closes )
Sergey M․ ff0f0b9172 [tube8] Fix extraction (Closes )
Sergey M․ 97b570a94c [generic] Improve rtl.nl embeds detection (Closes )
Sergey M․ a9d56c6843 [rtlnl] Improve _VALID_URL ()
Sergey M․ f98470df69 [bilibili] Fix FutureWarning
Jaime Marquínez Ferrándiz eb8be1fe76 [rtbf] Extract all formats (closes )
Yen Chi Hsuan 7ebd5376fe [nfl] Relax _VALID_URL (fixes )
Jaime Marquínez Ferrándiz 70219b0f43 [youtube:playlist] Use an iterator for the entries (closes )
So that '--playlist-end' downloads only the required pages.
Sergey M․ bd5bc0cd5a [theplatform] Check for /select/media URLs first ()
Sergey M․ 6e054aacca [theplatform] Take care of /select/media URLs (Closes )
Sergey M․ 9d581f3d52 [cbs] Extract display_id
Sergey M․ 9bf99891d0 [cbs] Add support for colbertlateshow (Closes )
Sergey M․ d9cf48e81e [spiegeltv] Extract all formats and prefer hls (Closes )
Yen Chi Hsuan e1b9322b09 [youtube] Restricter DASH signature pattern
A problematic DASH url is:
https://manifest.googlevideo.com/api/manifest/dash/mm/35/key/yt5/ip/140.112.247.145/ms/pm/mv/s/mt/1433794435/id/o-AD2Od_dsOlAUYPu03ZsVWKSbGEbCJJrMp9vnXGhnyRhd/mn/sn-aigllm7r/sparams/as%2Chfr%2Cid%2Cip%2Cipbits%2Citag%2Cmm%2Cmn%2Cms%2Cmv%2Cnh%2Cpl%2Cplayback_host%2Crequiressl%2Csource%2Cexpire/fexp/9406009%2C9406821%2C9407575%2C9408142%2C9408420%2C9408710%2C9409121%2C9409208%2C9412514%2C9412780%2C9413208%2C9413426%2C9413476%2C9413503%2C9415304%2C9415753/upn/viDQrs8SnmE/as/fmp4_audio_clear%2Cwebm_audio_clear%2Cfmp4_sd_hd_clear%2Cwebm_sd_hd_clear%2Cwebm2_sd_hd_clear/playback_host/r4---sn-aigllm7r.googlevideo.com/ipbits/0/requiressl/yes/pl/20/itag/0/source/youtube/expire/1433824806/nh/EAQ/signature/81ABE6391E351BA495F5B041B00FF1257A353318.1A6E48ABB74E8F4AE73CA2CB1F963FC34E33DEE7/sver/3/hfr/1
Yen Chi Hsuan 627b964825 [kickstarted] Extract thumbnails in embedded videos ()
Sergey M․ a55e36f48d [YoutubeDL] Handle out-of-range timestamps ()
Yen Chi Hsuan 01e21b89ee [noco] Skip invalid timestamps (closes )
Yen Chi Hsuan 788be3313d [cnet] Fix theplatform vid extraction (fixes )
Yen Chi Hsuan e1ec93304d [instagram:user] Truncate title to 80 characters ()
This is a workaround. Currently YoutubeDL.process_info() truncates
info_dict['title'] to 200 characters, but the implementation can't
handle wide characters.
Yen Chi Hsuan edb99d4c18 [instagram] Handling null values (fixes )
I didn't add the test case here because it takes too much time. (7
minutes on my machine)
Yen Chi Hsuan 68477c3dab [tlc] Fix test failure due to DiscoveryIE changes
Yen Chi Hsuan 65ba8b23f4 [discovery] Rewrite DiscoveryIE (fixes )
Discovery.com now uses a completely different approach for serving
videos. At least in both test cases brightcove are involved. However,
AMF support is necessary for these brightcove videos. As a result, I
try to extract videos from the info page ('?flat=1'). The downloaded
file can be different from the one in browsers.
Yen Chi Hsuan 621ed9f5f4 [common] Add note and errnote field for _extract_m3u8_formats
Yen Chi Hsuan b26733ba7f [brightcove] Allow single quotes in Brightcove URLs (fixes )
Sergey M․ 9836cfb8d6 [options] Clarify `--list-extractors` (Closes )
Sergey M․ 665b6c1236 Merge branch 'hlintala-ruutu'
Sergey M․ 9414338a48 [ruutu] Improve, make more robust and fix python 2.6 support
Jaime Marquínez Ferrándiz de390ea077 update: Use https for getting the version info (fixes )
Sergey M․ 717b0239fd Merge branch 'ruutu' of https://github.com/hlintala/youtube-dl into hlintala-ruutu
Hannu Lintala d00735a0c5 [ruutu] Don't use fallback for DASH and other non-HTTP urls
Yen Chi Hsuan c23d5ce926 Merge branch 'PeterDing-iqiyi'
Yen Chi Hsuan b5a3c7f109 [iqiyi] Cache encryption keys
Yen Chi Hsuan 9c5f685ef1 [iqiyi] Improve regex pattern again
Yen Chi Hsuan 08bb8ef201 [iqiyi] Unify get_format() and get_bid()
Yen Chi Hsuan 865ab62f43 [iqiyi] Make _VALID_URL more accurate
v_* urls are individual videos, while a_* urls are playlists, which are
not supported yet.
Yen Chi Hsuan 9948113590 [iqiyi] Add a multipart test case
Yen Chi Hsuan c4ee87022b [iqiyi] Change id for multipart videos
Yen Chi Hsuan ffba4edb06 [iqiyi] Improve some variable names and add download notes
Yen Chi Hsuan 958d0b659b [iqiyi] Reorder imports
Yen Chi Hsuan aacda28b28 [iqiyi] Give error message for assertion failures
Yen Chi Hsuan 29e7e0781b [iqiyi] Simplify and improve regex patterns
See the comments in 
Yen Chi Hsuan 7012620e2b [iqiyi] Remove format selection codes
Yen Chi Hsuan f1da861018 [iqiyi] PEP8
Naglis Jonaitis 05aa9c82d9 [sunporno] Fix view_count extraction
Naglis Jonaitis a9e58ecd3f [turbo] Improve description extraction
`og:description` is empty for some videos.
Hannu Lintala 223544552f [Ruutu] Add new extractor
Sergey M․ 3d8e9573a4 [youtube:channel] Improve channel id extraction ()
Naglis Jonaitis 54eb81a087 [pornovoisines] Improve average_rating extraction and update test case
Naglis Jonaitis c33c547d66 [izlesene] Avoid timestamp differences in tests due to DST
Naglis Jonaitis dfe7dd9bdb [izlesene] Unquote video URLs and simplify
Yen Chi Hsuan e8ac61e840 [qqmusic] Use meaningful variable names
Yen Chi Hsuan f00a650705 [qqmusic] Rearrange codes
Yen Chi Hsuan 4bde5ce992 Merge branch 'qqmusic-more-formats' of https://github.com/ping/youtube-dl into ping-qqmusic-more-formats
Yen Chi Hsuan d31573fa37 [teamcoco] Handle incomplete m3u8 URLs (fixes )
There are 2 TODOs. I don't know how to handle these cases correctly.
ping 8b8cde2140 [qqmusic] Set abr for mp3 formats
Philipp Hagemeister 0e805e782b release 2015.06.04.1
Philipp Hagemeister f5c78d118b release 2015.06.04
Yen Chi Hsuan 9d4f213f90 [qqmusic:toplist] List name and description are optional
Yen Chi Hsuan 168db222c6 Merge pull request from ping/qqmusic-toplist-fix
[qqmusic] Fix toplist extraction
Sergey M․ 3d6388e34e [tnaflix] Fix relative URLs (empflix)
Sergey M․ 3ce9bc712a [empflix] Fix typo
Sergey M․ e52c0bd0eb [tnaflix] Modernize
Sergey M․ 56c837ccb7 [tnaflix] Fix typo
ping 55e5841f14 [qqmusic] Extract additional formats (mp3-128, mp3-320)
ping ed15e9ba02 [qqmusic] Remove unused import
ping eedda32e6b [qqmusic] Fix toplist
Sergey M. d073055dcd Merge pull request from slava-sh/nova
[nova] Update
Slava Shklyaev e4ac7bb1e5 [nova] Revert "Fix extension extraction bug"
This reverts commit 9464a194db.
Yen Chi Hsuan 9bac8c57e3 Merge branch 'iqiyi' of https://github.com/PeterDing/youtube-dl into PeterDing-iqiyi
Sergey M․ 3153a2c98d [tvigle] Skip tests
Sergey M․ 15b74b94be [tvigle] Capture error message
Sergey M․ 687cb3ad35 [24video] Fix uploader extraction
Yen Chi Hsuan 8f94784124 [tumblr] Detect vid.me embeds (fixes )
Yen Chi Hsuan 23dd1fc74c [vidme] Always use the non-embedded page
For example, https://vid.me/Wmur contains more information than
https://vid.me/e/Wmur
Slava Shklyaev fa971259e6 [nova] Add a comment about html in description
Slava Shklyaev b0cda32f72 [nova] Fix Python 2.6 compatability issue
Slava Shklyaev 08b7968e28 [nova] Fix display_id extraction bug
Slava Shklyaev 4b5fe1349f [nova] Comply with review
Sergey M․ d23da75b32 [iprima] Fix description extraction
`og:description` does not contain actual description anymore.
Slava Shklyaev b5597738d4 [iprima] Comply with review
Slava Shklyaev bc03e58565 [iprima] Update
Slava Shklyaev a00234f1c5 [nova] Minor style improvement
Slava Shklyaev 34c0f95db2 [nova] Remove html tags from description
Slava Shklyaev fcb04bcaca [nova] Extract upload_date in some cases
Slava Shklyaev 9464a194db [nova] Fix extension extraction bug
Replace the hardcoded flv with determine_ext. Let rtmpdump parse the url.
Slava Shklyaev 9f4b9118cc [nova] Fix display_id extraction bug
Make id group non-greedy so that .html is not included in it.
Sergey M․ 60158217ef [nova] Add tv test
Sergey M․ 923e79e2e4 [nova] Add extractor
Naglis Jonaitis 866b296d0f [aftonbladet] Fix extraction and update _VALID_URL (Fixes )
Sergey M․ 47fd8c2f76 [patreon] Fix embeds extraction (Closes )
Sergey M․ 96b9690985 [imgur] Improve extraction
Sergey M․ df15ef8dab [YoutubeDL] Tweak select_format for video only media
Sergey M. 002c0fb511 Merge pull request from ivan/make-playlist-url
[youtube] Construct a playlist URL in case the page is missing one
Sergey M․ 7584e38ce4 [tvigle] Modernize
Sergey M․ eb47569f8a [tvigle] Add support for m3u8
Ivan Kozik d2a9de78df [youtube] Construct a playlist URL in case the page is missing one
This fixes jumping from user/channel -> playlist for some users like
https://www.youtube.com/user/BitcoinFoundation

This also removes the superfluous log message
"add --no-playlist to just download video VIDEOID"
when downloading a user/channel.
Sergey M․ c5138a7ce4 [extractor/generic] Clarify test comment
Sergey M․ c5fa81fe81 [extractor/generic] Put all direct link tests near to each other for better navigation
Sergey M․ a074e92296 [extractor/generic] Add test for large compressed media
Sergey M․ 1ddb9456c4 [extractor/generic] Use compat_urllib_parse_unquote for unquoting video_id and title from URL
Sergey M․ 58bde34a23 [extractor/generic] Force Accept-Encoding to any for extraction pass
Sergey M․ 339516072b [extractor/generic] Unescape video_id and title extracted from URL
Sergey M․ 931bc3c3a7 [YoutubeDL] Do not loose request method information
Yen Chi Hsuan a2d971309b [porn91] Use single quotes
Yen Chi Hsuan d05a1dbe70 [porn91] Catch daily limit error
Yen Chi Hsuan a80601f8d9 [porn91] Extract more info
Yen Chi Hsuan 1c22238756 [porn91] Simplify
Yen Chi Hsuan 9ff811c5cd [porn91] PEP8
Yen Chi Hsuan 1ebc05df91 Merge branch 'porn91' of https://github.com/PeterDing/youtube-dl into PeterDing-porn91
Sergey M․ 386bdfa698 [youtube:user] Workaround 35 pages limitation (Closes )
Naglis Jonaitis 1ae7ff771b [tubitv] Add error message for videos that require login ()
Naglis Jonaitis 5196b98897 [tubitv] Add new extractor (Closes )
Sergey M․ e6e63e91a7 [tf1] Extend _VALID_URL (Closes )
Sergey M․ b4dd98358f [vgtv] Properly handle lives
Sergey M․ 181c7053e3 [YoutubeDL] Make sure all formats have unique format_id
Sergey M․ 4d454c5e4b [vgtv] Check for inactive videos
Sergey M․ 5c2191a605 [vgtv] Skip wasLive hds (Closes )
Sergey M․ bba5bfc890 Merge branch 'ping-soompi'
Sergey M․ 1a5b77dc21 [crunchyroll] Fix python 3.2
Sergey M․ b2cf6543b2 [soompi] Improve and simplify
Sergey M․ 0385d64223 [crunchyroll] Extract subtitles extraction routine
pulpe 6ebdfe43e4 [tube8] fix extractor (fixes )