Commit Graph

7811 Commits (3a30508b943c044e5f684b703ff58ac352686f63)

Author SHA1 Message Date
fnord 53b8247cb5 NationalGeographic._VALID_URL: work site-wide
Closes .
Jaime Marquínez Ferrándiz 59db9f8018 [downloader/dash] Improve 'combine_url' (fixes )
In some videos the base_url already ends with '/'.
Yen Chi Hsuan b73b14f72c [viki] Rewrite dict selection codes
Yen Chi Hsuan 41597d9bed [viki] Fix description extraction (closes )
Jaime Marquínez Ferrándiz b37317d8b0 [generic] Unescape HTML escape sequences in redirect urls (fixes )
Jaime Marquínez Ferrándiz 87dc451108 [youtube] Don't use the DASH manifest from 'get_video_info' if 'use_cipher_signature' is True ()
Currently they give a 403 Forbidden error.
Jaime Marquínez Ferrándiz ca4456eda8 [tagesschau] Remove unused import
Sergey M․ 993df6bc22 [prosiebensat1] Modernize
Sergey M․ 61be92e26a [prosiebensat1] Recognize DRM protected videos ()
Sergey M․ c59b61c0da [viki] Fix height (Closes )
Sergey M․ 3e214851a4 [tagesschau] Improve tests
Sergey M․ a47b602b08 [tagesschau] Add support for audio
Sergey M․ 948199deac [tagesschau] Relax _VALID_URL and simplify
Sergey M․ c356620ec1 Merge branch 'patch/enhance-tagesschau-regex' of https://github.com/rohieb/youtube-dl into rohieb-patch/enhance-tagesschau-regex
Sergey M․ c7620992d2 [pbs] No need to escape colon
Sergey M․ ce1bafdce9 [pbs] Clean up title construction rationale
Sergey M. 9872e588c8 Merge pull request from atomicdryad/pr-fix_pbs_titles
pbs: fix vague 'Full Episode' titles; prepend name of show
Sergey M․ d609edf4f1 [udemy] Handle already-logged-in scenario (Closes )
Yen Chi Hsuan 3a99d321a8 Merge pull request from jaimeMF/dash_no_live
[YoutubeDL] don't default to 'bestvideo+bestaudio/best' for live videos
Yen Chi Hsuan 40101dc311 [lecture2go] Make optional fields non-fatal
Yen Chi Hsuan e9c6deffee [lecture2go] Add more metadata fields
Yen Chi Hsuan 9c29bc69f7 [utils] Improve parse_duration
Now dots are parsed. For example '87 Min.'
Yen Chi Hsuan 1e12429564 [lecture2go] Update _TEST
Yen Chi Hsuan 795704f0f1 [lecture2go] Support more formats
Yen Chi Hsuan 981b9cdc8c [lecture2go] Improve some regular expressions
Yen Chi Hsuan 3f724339db Merge branch 'lecture2go' of https://github.com/nichdu/youtube-dl into nichdu-lecture2go
Sergey M․ c84683c88b [viewster] Strip titles
Sergey M․ b68a2613f8 [viewster] Rewrite for new API (Closes )
Jaime Marquínez Ferrándiz 28afa6e77a [snagfilms] Don't use '_' as a variable that is used
Sergey M․ 496ce6b349 [snagfilms] Improve m3u8 extraction (Closes )
Philipp Hagemeister ce9512b78b release 2015.07.21
Sergey M․ 4eb59a6b1c [options] Fix a typo ()
Jaime Marquínez Ferrándiz 7c80519cbf [youtube] Extract start_time
From the 't=*' in the url.
Currently youtube-dl doesn't use the value, but it was requested for the mpv plugin.
Jaime Marquínez Ferrándiz 8250c32f49 [YoutubeDL] don't default to 'bestvideo+bestaudio/best' for live videos
Doesn't work currently.
Jaime Marquínez Ferrándiz 2fe1ff8582 [youtube] Set 'is_live'
Jaime Marquínez Ferrándiz 17ee98e1a5 [youtube] Fix upload_date in test
Jaime Marquínez Ferrándiz 2ee8f5d80f Merge remote-tracking branch 'yan12125/download-dash-segments' ()
Yen Chi Hsuan 3f302bca8c [tumblr] Improve downloading notes
Yen Chi Hsuan c909e5820e [tumblr] Delegate to GenericIE for non-tumblr videos
Fixes 
Yen Chi Hsuan a1b85269a4 [extractor/generic] Support vid.me embeds
Sergey M․ faa1f83ab4 [twitch:stream] Improve _VALID_URL (Closes )
Sergey M․ 308c505c3d [francetv] Improve _VALID_URL
fnord 0eacd2aaae less clunky if statement
Sergey M․ 18ae46ad4b [vice] Modernize
Sergey M․ 65c2b21df1 [vice] Make test only matching
Sergey M․ 772acaf31f [vice] Do not capture unused groups in _VALID_URL
Sergey M. f8d0745e27 Merge pull request from atomicdryad/pr-fixvice
fix/support news.vice.com
Sergey M․ d719c6a5ab [ard:mediathek] Add test for direct mp4
Sergey M․ 769efa16af [ard:mediathek] Remove dead test
Sergey M․ 86b4e98ac6 [ard:mediathek] Add audio test
Sergey M․ 3bf8c316a6 [sportschau] Reimplement in terms of ard extractor
Sergey M․ e37c92ec6d [ard] Extract all formats
fnord a5dd9a0c5d pbs: fix vague 'Full Episode' titles; prepend name of show
Many videos are given the title 'Full Episode' or 'Episode x', etc;
in the info json 'title' is the episode title (for oneshots this is
'Full Episode') while the program name is 'program':{'title'. This
changes the title from '(episode title)' to
 '(program name) - (episode title)'

The following urls demonstrate useless titles

    http://video.pbs.org/video/2365367186/
    http://video.pbs.org/video/2365519307/
    http://video.pbs.org/video/2365527039/
    http://video.pbs.org/video/2365530605/

Before change

    Full Episode
    Episode 5 | Preview
    Season 3 | Episode 4 Preview
    July 17, 2015

After change

    To Catch A Comet - Full Episode
    The Crimson Field - Episode 5 | Preview
    Last Tango in Halifax - Season 3 | Episode 4 Preview
    Charlie Rose The Week - July 17, 2015
fnord 7a4a945f13 fix/support news.vice.com
Sergey M․ 1d18e26eca [francetv] Remove unused import
Sergey M․ ac4b8df5e4 [francetv] Fix embed test
Sergey M․ 3bc9fb5889 [francetv] Update jt test
Sergey M․ 632cbb8efa [francetv] Fix f4m extraction completely
Sergey M․ 789a12aaaf [francetv] Restore support for jt videos
Sergey M․ ecdbe09e10 [francetv] Fix f4m extraction
Zach Bruggeman 1dc31c2786 [appleconnect] Add new extractor (fixes )
Closes .
Jaime Marquínez Ferrándiz 32470bf619 [sportschau] Improve title extraction
The html '<title>' ends with '- sportschau.de', which shouldn't be part of the title.
Jaime Marquínez Ferrándiz 8b61bfd638 [sportschau] skip download in test
slangangular 8a7a208905 [sportschau] Add support for sportschau.de
Closes .
Jaime Marquínez Ferrándiz 0215103e92 [francetv] Remove unused import
Yen Chi Hsuan c2d1be8981 [iqiyi] Add skip_download to reduce network traffic
MD5 checksums are commented out in
36068ae019, and actual downloads are not
necessary anymore.
Roland Hieber 726adc43ec [tagesschau] set description=None for empty descriptions
Roland Hieber 3c6ae8b59e [tagesschau] add support for Bericht aus Berlin videos
Roland Hieber c51bc70e0f [tagesschau] fix test which links to nonexisting video
Roland Hieber e89d7e3029 [tagesschau] add support for more video types
I found that currently only tagesschau videos are played. There are some
more shows hosted on tagesschau.de (see [0] for example) which are
easily playable by adjusting the regex. So this patch adds support for:
* tagesthemen
* tagesschau vor 20 Jahren
* tagesschau (mit Gebärdensprache)
* nachtmagazin

Note that some videos don't provide a description, so in order for the
tests to succeed, an ExtractorError needs to get caught.

[0]: http://www.tagesschau.de/multimedia/video/videoarchiv2~_date-20150714.html
Yen Chi Hsuan 4b0f45f667 Merge branch 'sceext2-fix-iqiyi-2015-07-17'
Yen Chi Hsuan 36068ae019 [iqiyi] Comment out some MD5 sums
The value is different on Travis CI server
remitamine 9d681c2bb3 remove unnecessary group name
remitamine 3af1fac7b0 [dcn] Add new extractor
Yen Chi Hsuan 761ee0d827 [iqiyi] Remove unused imports
Yen Chi Hsuan fb8bc3f818 Merge branch 'fix-iqiyi-2015-07-17' of https://github.com/sceext2/youtube_dl into sceext2-fix-iqiyi-2015-07-17
Sergey M․ 826a7da808 [rtlnl] Avoid episodes completely (Closes )
Sergey M․ cbd55ade68 [extractor/generic] Add support for francetv embeds
Sergey M․ 5705ee6ef8 [francetv] Fix duration extraction
Sergey M․ 3f5c6d0c1b [francetv] Add support for embeds and clean up _VALID_URL
Sergey M․ e58066e244 [vk] Add age restricted video test for reference
Sergey M․ ee48b6a88f [vk] Capture error message
Sergey M․ 9ac09ed4de [bliptv] Remove unused import
Sergey M․ 22603348aa [compat] Fix _asciire
Sergey M․ fec73daaa3 [vk:uservideos] Revert orderedSet
Sergey M․ c6b68648f4 [bilibili] Show georestriction error
Sergey M․ 1ecb5d1d83 [vk] Clarify extractor names
Sergey M․ dc786d3db5 [vk:uservideos] Improve extraction
Sergey M․ 74fe23ec35 [extractor/common] Style
Philipp Hagemeister b0bff54b08 release 2015.07.18
sceext 1b541d8d6e [iqiyi] fix iqiyi (2015-07-17), update the md5 salt (enc_key) to iqiyi latest (2015-07-17) flash player.
Sergey M․ f29ac588dd [nationalgeographics] Fix extraction (Closes )
Sergey M․ 0696667734 [twitch:stream] Clarify channel_id reassignment rationale ()
Sergey M․ 1793d71db6 [twitch:stream] Fix channel_id in different case (Closes )
Sergey M․ 4211e1941b [videomega] Add shortcut to _VALID_URL
Sergey M․ 4bdfef5a18 [videomega] Add tests
Sergey M․ 8a37f53685 [videomega] Revert iframe URL
Sergey M․ 4e1ad6e9a8 [videomega] Extend _VALID_URL and improve extraction (Closes )
Sergey M․ 3c283a381e [sbs] Simplify
Sergey M․ dac4d5be12 Merge branch 'sbs_website_changes' of https://github.com/seamusphelan/youtube-dl into seamusphelan-sbs_website_changes
Sergey M․ 530857182d [bliptv] Add test with missing duration
Sergey M․ 9441f77faa [bliptv] Use xpath for extraction and fix test
Sergey M․ 3cc8b4c327 [compat] Fix missing _asciire on python 2.6
Sergey M․ 6b19647d57 [veehd] Use compat_urllib_parse_unquote
Sergey M․ 7bd42d0d96 [openfilm] Fix compat_urllib_parse_unquote_plus import
Sergey M․ c60e8cfaf7 [ehow] Simplify
Sergey M․ 7fd002c006 [youtube] Use compat_urllib_parse_unquote and compat_urllib_parse_unquote_plus
Sergey M․ db6c50f109 [ynet] Use compat_urllib_parse_unquote_plus
Sergey M․ aa4789d632 [xvideos] Use compat_urllib_parse_unquote
Sergey M․ ee8de13e14 [xtube] Use compat_urllib_parse_unquote
Sergey M․ 7dde5f6a8d [xnxx] Use compat_urllib_parse_unquote
Sergey M․ 736f003f2e [xbef] Use compat_urllib_parse_unquote
Sergey M․ 47af21e8f1 [spankwire] Use compat_urllib_parse_unquote
Sergey M․ 605cbef653 [pornhub] Use compat_urllib_parse_unquote and compat_urllib_parse_unquote_plus
Sergey M․ 388ad0c05c [playvid] Use compat_urllib_parse_unquote and compat_urllib_parse_unquote_plus
Sergey M․ 2ebbb6f1f7 [photobucket] Use compat_urllib_parse_unquote
Sergey M․ d54f1c7477 [openfilm] Use compat_urllib_parse_unquote_plus
Sergey M․ b78f5ec4c3 [odnoklassniki] Use compat_urllib_parse_unquote
Sergey M․ 9fd3bf04b7 [myvideo] Use compat_urllib_parse_unquote
Sergey M․ e97bb3de83 [mofosex] Use compat_urllib_parse_unquote
Sergey M․ c2daf8dfa4 [mixcloud] Use compat_urllib_parse_unquote
Sergey M․ 09b718c439 [mitele] Use compat_urllib_parse_unquote
Sergey M․ c177bb3a50 [metacafe] Use compat_urllib_parse_unquote
Sergey M․ 977a247a06 [malemotion] Use compat_urllib_parse_unquote
Sergey M․ 899a3e2f13 [karaoketv] Use compat_urllib_parse_unquote_plus
Sergey M․ 8ee4ecb48d [infoq] Use compat_urllib_parse_unquote
Sergey M․ f7e6f7fa23 [extractor/generic] Use compat_urllib_parse_unquote
Sergey M․ 1f80e360fc [gamespot] Use compat_urllib_parse_unquote
Sergey M․ d7011316d0 [facebook] Use compat_urllib_parse_unquote
Sergey M․ d3671b344f [ehow] Use compat_urllib_parse_unquote
Sergey M․ a60cccbf9f [crunchyroll] Use compat_urllib_parse_unquote
Sergey M․ 3e72f5f10e [ceskatelevize] Use compat_urllib_parse_unquote
Sergey M․ b94b78971c [bet] Use compat_urllib_parse_unquote
Sergey M․ 4d08161ac2 [compat] Mention unquote_plus
Sergey M․ aa99aa4e85 [compat] Add compat_urllib_parse_unquote_plus
Sergey M․ 13fc7f3a05 Merge branch 'atomicdryad-pr-crashfix_compat_urllib_unquote'
Sergey M․ 5513967926 [compat] Simplify and use latest cpython 3 code
Sergey M. eacd875f3b Merge pull request from remitamine/howstuffworks
[howstuffwoks] fix _VALID_URL regex
remitamine c4fe07c7af match only the test url
remitamine 1186e3f91a do not capture a group
remitamine f354385bf5 Add test for urls without a number
remitamine cabe001590 [howstuffwoks] fix _VALID_URL regex
Sergey M․ 89f691e141 Merge branch 'pr-crashfix_compat_urllib_unquote' of https://github.com/atomicdryad/youtube-dl into atomicdryad-pr-crashfix_compat_urllib_unquote
fnord 593b77064c Don't forget trailing '%'
fnord 9fefc88656 fix TestCompat test_all_present
fnord a3bfddfa5e bbc.py: correct syntax
fnord 36da48798a handle titles and captions set to ''
fnord a0f28f90fa remove kebab
fnord 851229a01f remove debugprint
fnord c9c854cea7 replace old compat_urllib_parse_unquote with backport from python3's function
* required unquote_to_bytes function ported as well
    (uses .decode('hex') instead of dynamically populated _hextobyte global)
  * required implicit conversion to bytes and/or unicode in places due to
    differing type assumptions in p3
Yen Chi Hsuan a38436e889 [extractor/common] Add 'transform_source' parameter to _extract_f4m_formats()
Sergey M․ 23fc384f2c [npo] Compound title
Sergey M․ 1540119723 [npo] Remove unused imports
Sergey M․ 574f42d79a [vpro] Improve playlist extraction
Sergey M․ 536b0700b0 [npo] Allow missing description
Sergey M․ 5ba761eb85 [npo] Prefer aflevering_titel over titel
Sergey M․ 611ac379bb [vpro] Fix extraction and add support for vpro playlists
Sergey M․ 03f32a7ead [wnl] Add extractor for omroepwnl playlists
Sergey M․ 50ea2bb20d [npo] Update test
Sergey M․ 525daedd5a [npo] Add support for omroepwnl fragments
Sergey M․ e118031ef8 [npo] Extend _VALID_URL to support ntr.nl (Closes )
fnord 45eedbe58c Generic: use compat_urllib_parse_unquote to prevent utf8 mangling
of the entire page in python 2.

-requires- fixed compat_urllib_parse_unquote

example - the following will save with a mangled playlist title,
 instead of the kanji for 'tsunami'. This affects all utf8encoded
 urls as well

youtube-dl -f18 -o '%(playlist_title)s-%(title)s.%(ext)s' \
  61c14c1e3a/tsunami.html
fnord e37c932fca compat_urllib_parse_unquote: crash fix: only decode valid hex
on python 2 the following has a { "crash_rate": "100%" } of the time
as it tries to parse '" ' as hex.
Sergey M․ ab9b890b52 [prosiebensat1] Clarify test purpose
Sergey M․ 31c746e5dc [extractor/common] Keep going in some media_url is missing
Sergey M․ f01f731107 [prosiebensat1] Use generic f4m manifest extraction
Sergey M․ 70f0f5a8ca [extractor/common] Recursively extract child f4m manifests
Sergey M․ cc357c4db8 [extractor/common] Properly handle full URLs
Sergey M․ 97f4aecfc1 [extractor/common] Handle malformed f4m manifests
Sergey M․ 2af0f87c8b [prosiebensat1] Fix extraction (Closes )
Sergey M․ 6c1b0c0ed2 [dfb] Extract formats
Sergey M․ ddcdc684e2 [dfb] Extract display_id and modernize
Sergey M․ eae89f92e6 [dfb] Update test
Jaime Marquínez Ferrándiz 01d115b06b [jeuxvideo] Relax _VALID_URL (fixes )
Sergey M․ 79057965a8 [udemy] Remove superfluous field
Sergey M․ dcd4d95c8e [udemy] Fix authentication (Closes )
Sergey M․ cf61d96df0 [extractor/common] Add _form_hidden_inputs
Sergey M․ f8da79f828 [extractor/common] Improve _form_hidden_inputs and rename to _hidden_inputs
Jaime Marquínez Ferrándiz 9750e7d70e [postprocessor/ffmpeg] Don't use '[youtube] ...' in messages
Because it can be used for other extractors.
cazulu 50aa2bb6b9 [dailymotion] Extract duration (closes )
Sergey M․ 1d1dd597ed [dramafever] Extract srt (Closes )
Yen Chi Hsuan cfe5537ee5 [myspass] Fix extraction (closes )
Sergey M․ 6dfa0602f0 [nowtv] Fix extraction (Closes )
Seamus Phelan 75a40b2251 [SBS] fixes due to website changes
Sergey M․ 28fb109ed0 [rds] Improve _VALID_URL
Sergey M․ b6ea9ef21a [rds] Improve
Sergey M․ b8dd44baa9 Merge branch 'rdsca' of https://github.com/fstirlitz/youtube-dl into fstirlitz-rdsca
Sergey M․ c4f1fde75b [nrk:tv] Add format id prefixes
Sergey M․ 667170e2c7 [nrk:tv] Add support for radio URLs (Closes )
Sergey M․ 53429e6551 [onionstudios] Fix extraction
felix ac8f97f2b3 [rdsca] New extractor
Sergey M․ 369e195a44 Handle postprocessor_args similarly to external_downloader_args
Sergey M․ 15006fedb9 [postprocessor/ffmpeg] Spread postprocessor args usage on all ffmpeg extractors
Sergey M․ e35b23f54d [postprocessor/common] Improve postprocessor args fetching and clarify doc
Sergey M․ f72b0a6032 Revert xvid to avi and make docs to be similar to existing external downloader option
Sergey M․ ac9ed061ec Merge branch 'master' of https://github.com/aurium/youtube-dl into aurium-master
Sergey M․ d919fa3344 [vk] Handle access denied error
Sergey M․ 79913fde35 [vk] Add list id to info_url
Sergey M․ da634d0a8b Merge branch 'dufferzafar-webofstories'
Sergey M․ fac54cb426 [webofstories:playlist] Improve and add test