Commit Graph

11895 Commits (c131fc3372c4fc69434dbb7c79935c1587beff20)

Author SHA1 Message Date
Yen Chi Hsuan 2ecbd2ad6f
[chirbit:profile] Fix extraction
Yen Chi Hsuan 62a0b86e4f
[carambatv] Fix extraction
The video requested in  now has videomore embeds.
Yen Chi Hsuan 146969e05b
[videomore] Support <iframe> embed videos
Seen in CarambaTVPage
Yen Chi Hsuan e2004ccaf7
[canalplus] Fix video_id and update _TESTS
Some tests are gone, and some redirect to different videos
Yen Chi Hsuan a5f8473145
[cbsinteractive] Fix extraction for cnet.com
Philip Xu b7f59a3bf6 [huajiao] Add new extractor
Yen Chi Hsuan 580d411931
[parliamentliveuk] Recognize lower case URLs
Closes 

Seems parliamentliveuk matches URLs case-insentive. For example this URL
also works:
http://parliamentlive.tv/EvEnt/Index/3F24936f-130f-40bf-9a5d-b3d6479da6a4
Sergey M․ 5c4bfd4da5
release 2016.10.12
Sergey M․ bcd6276520
[downloader/common] Remove debug output
Sergey M․ 591e384552
[streamable] Remove debug output
Yen Chi Hsuan 9feb1c9731
[dailymotion] Fix extraction and update _TESTS
Closes 

Seems all videos use player V5 syntax now
Yen Chi Hsuan a093cfc78b
[vimeo:review] Fix extraction ()
Now Vimeo Review videos uses React. Thanks @davekaro for analyzing the
problem!
Yen Chi Hsuan cea364f70c
[extractor/common] Support HTML media elements without child nodes
Yen Chi Hsuan 55642487f0
[nhl] Skip invalid m3u8 formats (closes )
Yen Chi Hsuan 3d643f4cec
[hbo] Add HBOEpisodeIE ()
Yen Chi Hsuan c452e69d3d
[footyroom] Fix extraction and update _TESTS (closes )
Yen Chi Hsuan 555787d717
[streamable] Add helper for extracting embedded videos
Yen Chi Hsuan f165ca70eb
[abc.net.au:iview] Fix for non-series videos (closes )
Yen Chi Hsuan 27b8d2ee95
[hbo] Add display_id and another test ()
Yen Chi Hsuan 71cdcb2331
[hbo] Support episode pages (closes )
Yen Chi Hsuan 176006a120
[allocine] Fix for /video/ videos (closes )
Yen Chi Hsuan 65f4c1de3d
[allocine] Fix extraction (closes )
I change the URL of the third test case, because now the original URL
does not contain a video anymore, and there's no easy to get the real
URL from the /film/ one.
Yen Chi Hsuan b0082629a9
[nextmedia] Support action news (動新聞) on Apple Daily
Déstin Reed 2b51dac1f9
[slutload] Fix test and simplify
Sergey M․ f68901e50a
[reverbnation] Eliminate code duplication in thumbnails extraction
Déstin Reed 3adb9d119e
[reverbnation] Modernize
Remita Amine 1dd58e14d8 [lego] improve info extraction and bypass geo restriction(closes )
Sergey M․ dd4291f729
release 2016.10.07
Sergey M․ f475e88121
[vimeo] PEP 8
[ci skip]
Remita Amine 3c6b3bf221 [iprima] detect geo restriction
Yen Chi Hsuan 38588ab977
[facebook] Fix for new handleServerJS syntax (closes )
According to the dump file in , handleServerJS() now accepts
an optional second argument. It's a string from available dump files.
Yen Chi Hsuan 85bcdd081c
[extractors] Add MmsIE
Yen Chi Hsuan 9dcd6fd3aa
[generic,commonprotocols] Move mms suuport from GenericIE
And use _generic_* helpers in those extractors
Yen Chi Hsuan 98763ee354
[extractor/common] Add id and title helpers for generic IEs
Yen Chi Hsuan 3d83a1ae92
[generic] Support direct MMS links (closes )
Sergey M․ 09b9c45e24
[generic] Add support for multiple vimeo embeds (Closes )
Remita Amine 33898fb19c [nzz] Add new extractor()
Remita Amine 017eb82934 [npo] detect geo restriction
Sergey M․ b1d798887e
[npo] Add support for 2doc.nl (Closes )
Remita Amine 185744f92f [lego] Add new extractor(closes )
Remita Amine 7232e54813 [tonline] Add new extractor()
Sergey M․ 6eb5503b12
[techtalks] Relax _VALID_URL
Aleksander Nitecki 539c881bfc
[techtalks] Allow URL-s with name part omitted.
Sergey M․ c1b2a0858c
[youtube:live] Extend _VALID_URL (Closes )
Remita Amine 215ff6e0f3 [theweatherchannel] Add new extractor(closes )
Déstin Reed dcdb292fdd
Unify coding cookie
Remita Amine c1084ddb0c [thisoldhouse] Add new extractor(closes )
Sergey M․ ee5de4e38e
[nhl] Add support for wch2016.com (Closes )
Yen Chi Hsuan 25291b979a Merge pull request from TRox1972/pornoxo_improve
[pornoxo] Use JWPlatform to improve metadata extraction
Déstin Reed 567a5996ca [pornoxo] Use JWPlatform to improve metadata extraction
Sergey M․ 6c152ce20f
release 2016.10.02
Yen Chi Hsuan 703b3afa93
[amcnetworks] Skip a restricted _TEST
Yen Chi Hsuan 99ed78c79e
[jwplatform] Support DASH streams
Yen Chi Hsuan fd15264172
[jwplatform] Support old-style jwplayer playlists
Yen Chi Hsuan bd26441205
[utils] Fix xattr error handling
Yen Chi Hsuan b19e275d99
[__init__] Fix lost xattr if --embed-thumbnail used
Reported at
https://github.com/rg3/youtube-dl/issues/9054#issuecomment-250451823
Sergey M․ f6ba581f89
[byutv:event] Add extractor
Sergey M․ 6d2549fb4f
[byutv] Fix id and display id
Déstin Reed 4da4516973
[byutv] Rely on _match_id and _parse_json
Sergey M․ e1e97c2446
[periscope:user] Fix extraction (Closes )
Yen Chi Hsuan 53a7e3d287
[utils] Support xattr as well as pyxattr
Closes 

There are two xattr packages in Python, pyxattr [1] and xattr [2]. They
have different APIs.

In old days pyxattr supports Linux only and xattr supports Linux, Mac,
FreeBSD and Solaris, and pyxattr supports Linux only. Recently pyxattr
adds support for Mac OS X. [3]

An old version of [2] is shipped with Mac OS X. However, some Linux
distributions have pyxattr only, for example PLD-Linux [4] and old Arch
Linux. [5] As a result, supporting both is the way to go.

[1] https://github.com/iustin/pyxattr
[2] https://github.com/xattr/xattr
[3] https://github.com/iustin/pyxattr/pull/9
[4] https://github.com/rg3/youtube-dl/issues/5498
[5] https://git.archlinux.org/svntogit/community.git/commit/?id=427c4c76401e386d865ccddea4fbfdc74df80492
    https://git.archlinux.org/svntogit/community.git/commit/?id=59b40da7b69622a6761d364a8b07909e9cccaa56
    python-xattr is added on 2016/06/29 while pyxattr is there for more
    than 6 years
Yen Chi Hsuan d54739a2e6
[downloader/http] xattr values should be bytes
Yen Chi Hsuan 63e0fd5bcc Merge pull request from TRox1972/criterion_match_id
[criterion] Rely on _match_id, improve regex and add thumbnail to test
Déstin Reed 9c51a24642 [criterion] Rely on _match_id, improve regex and add thumbnail to test
Yen Chi Hsuan 9bd7bd0b80
[twitch] Skip a 404 test
Yen Chi Hsuan 4a76b73c6c Merge pull request from TRox1972/clubic_match_id
[clubic] Rely on _match_id and _parse_json
Yen Chi Hsuan e295618f9e
[dctp] Fix extraction (closes )
Yen Chi Hsuan d7753d1948
[downloader/http] Use write_xattr function for --xattr-set-filesize
Déstin Reed eaf9b22f94 [clubic] Rely on _match_id and _parse_json
Sergey M․ a1001f47fc
[instagram] PEP 8
Déstin Reed 1609782258
[Instagram] Extract video dimensions
Sergey M․ de6babf922
[tvland] Extend _VALID_URL (Closes )
Sergey M․ b0582fc806
[vgtv] Add support for tv.aftonbladet.se (Closes )
Sergey M․ af33dd8ee7
[aftonbladet] Remove extractor
Sergey M․ 70d7b323b6
[vk] Improve view count extraction
Sergey M․ a7ee8a00f4
[vk] Extract timestamp (Closes )
Sergey M․ c6eed6b8c0
[utils] Lower priority for rare date formats and add tests
Kacper Michajłow 3aa3953d28
[vk] Fix date and view count extraction.
Yen Chi Hsuan efa97bdcf1
Move write_xattr to utils.py
There are some other places that use xattr functions. It's better to
move it to a common place so that others can use it.
Sergey M․ 475f8a4580
[vk] Add support for running live streams (Closes )
Sergey M․ 93aa0b6318
[vk] Add support for finished live streams ()
Yen Chi Hsuan 0ce26ef228 Merge pull request from TRox1972/instagram_comments
[Instagram] Extract comments
Yen Chi Hsuan 0d72ff9c51
[leeco] Recognize more Le Sports URLs ()
Déstin Reed a56e74e271 [Instagram] Extract comments
Sergey M․ f533490bb7
[ketnet] Extract mzsource formats ()
Remita Amine 8bfda726c2 [limelight:media] improve http formats extraction
Sergey M․ 8f0cf20ab9
release 2016.09.27
Sergey M․ cdfcc4ce95
[mtv] Improve _VALID_URL
Kacper Michajłow e384552590
[vk] Add support for dailymotion embeds
Fixes 
Sergey M․ 1a2fbe322e
[periscope] Treat timed_out state as finished stream
Sergey M․ f9dd86a112
[npo] Clarify IE_NAMEs (Closes )
Remita Amine 2342733f85 fix tests related to 1978540a5122c53012e17a78841f3da0df77fd34(closes )
Remita Amine 93933c9819 [awaan:video] fix test(closes )
Yen Chi Hsuan d75d9e343e
[einthusan] Fix extraction (closes )
Sergey M․ 72c3d02d29
[promptfile] Improve and modernize
Ondřej Bárta d3dbb46330
[promptfile] Fix extraction (Closes )
Sergey M․ fffb9cff94
[kaltura] Speed up embed regexes ()
Sergey M․ 2d5b4af007
[extractors] Add import for anderetijden extractor
Sergey M․ 5742c18bc1
[npo] Add support for anderetijden.nl (Closes )
Sergey M․ ddb19772d5
[vpro] Fix playlist title extraction and update tests
Sergey M․ a3d8b38168
[npo] Generalize playlist extractors
stepshal 2d3d29976b
[youtube] Change test URLs from http to https
Sergey M․ 493353c7fd
[prosiebensat1] Add support for advopedia
Sergey M․ 0a078550b9
[prosiebensat1] Improve _VALID_URL
Sergey M․ f92bb612c6
[mwave] Relax _VALID_URLs (Closes , closes )
Sergey M․ ddde91952f
[prosiebensat1] Fix playlist support (Closes )
Sergey M․ 63c583eb2c
[prosiebensat1] Add support for sat1gold ()
Remita Amine 7fd57de6fb [cbsnews:livevideo] fix extraction and extract m3u8 formats
Remita Amine e71a450956 [common] add hdcore sign to akamai f4m formats
Remita Amine 27e99078d3 [brightcove:new] add support for live streams
Remita Amine 6f126d903f [download/hls] Delegate downloading to ffmpeg for live streams
Sergey M․ 7518a61d41
[soundcloud] Fix typo in playlist base class name
Sergey M․ 8e45e1cc4d
[soundcloud] Generalize playlist entries extraction ()
Yen Chi Hsuan f0bc5a8609
[twitter] Support Periscope embeds (closes )
Also update _TESTS
Remita Amine a54ffb8aa7 [mtv] add common IE_NAME prefix for MTVIE and MTVVideoIE
Remita Amine 8add4bfecb [mtv] add support for new website urls(closes )(closes )
Yen Chi Hsuan 0711995bca
[openload] Support subtitles (closes )
Yen Chi Hsuan 5968d7d2fe
[extractor/common] Improved support for HTML5 subtitles
Ref: 

In a strict sense, <track>s with kind=captions are not subtitles. [1]
openload misuses this attribute, and I guess there will be more
examples, so I add it to common.py.

Also allow extracting information for subtitles-only <video> or <audio>
tags, which is the case of openload.

[1] https://www.w3.org/TR/html5/embedded-content-0.html#attr-track-kind
Sergey M․ e6332059ac
release 2016.09.24
Sergey M․ 24628cf7db
[soundcloud:playlist] Provide video id for playlist entries (Closes )
Sergey M․ 71ad00c09f
[prosiebensat1] Add support for kabeleinsdoku (Closes )
Remita Amine 45cae3b021 [cbs] extract info from thunder videoPlayerService(closes )
Yen Chi Hsuan 4ddcb5999d
[openload] Fix extraction (closes , closes )
Thanks to @daniel100097 for providing a working version
Yen Chi Hsuan e3d6bdc8fc
[ustream] Support HLS streams (closes )
Sergey M․ 0a439c5c4c
[udemy] Stringify video id
Remita Amine 1978540a51 [ooyala] extract all hls formats
Sergey M․ 12f211d0cb
[videomore] Fix embed regex
Remita Amine 3a5a18705f [adobepass] add support MSO that depend on watchTVeverywhere(closes )
Remita Amine 1ae0ae5db0 [cartoonnetwork] add support Adobe Pass auth
Sergey M․ f62a77b99a
[soundcloud] Modernize
coolsa 4bfd294e2f
[soundcloud] Extract license metadata
Remita Amine e33a7253b2 [fox] add support for Adobe Pass auth(closes )
Remita Amine c38f06818d add support for Adobe Pass auth in tbs,tnt and trutv extractors(fixes )(closes )(closes )
Sergey M․ cb57386873
release 2016.09.19
Sergey M․ 70b4cf9b1b
[crunchyroll] Check if already logged in (Closes )
Sergey M․ cc764a6da8
[twitch:stream] Remove fallback to profile extraction when stream is offline
Main page does not contain profile videos anymore
Yen Chi Hsuan d8dbf8707d
[thisav] Improve title extraction (closes )
I didn't add a test case as the one in  looks like a copyrighted
product.
Sergey M․ a1da888d0c
[vyborymos] Improve station info extraction
Sergey M․ 3acff9423d
release 2016.09.18
Sergey M․ 14ae11efab
[vyborymos] Add extractor (Closes )
Sergey M․ 190d2027d0
[xfileshare] Add title regex for streamin.to and fallback to video id (Closes )
Sergey M․ 26394d021d
[globo:article] Add support for multiple videos (Closes )
Sergey M․ 30d0b549be
[extractor/common] Add manifest_url for hls and hds formats
Sergey M․ 86f4d14f81
Refactor fragments interface and dash segments downloader
- Eliminate segment_urls and initialization_url
+ Introduce manifest_url (manifest may contain unfragmented data in this case url will be used for direct media URL and manifest_url for manifest itself correspondingly)
* Rewrite dashsegments downloader to use fragments data
* Improve generic mpd extraction
Sergey M․ 21d21b0c72
[svt] Fix DASH formats extraction
Sergey M․ b4c1d6e800
[extractor/common] Expose fragments interface for dashsegments formats
Sergey M․ a0d5077c8d
[extractor/common] Introduce fragments interface
Yen Chi Hsuan 584d6f3457
[thisav] Recognize jwplayers (closes )
Yen Chi Hsuan e14c82bd6b
[jwplatform] Use js_to_json to detect more JWPlayers
Sergey M․ c51a7f0b2f
[franceinter] Fix upload date extraction
Remita Amine d05ef09d9d [mangomolo] fix domain regex
Remita Amine 30d9e20938 [postprocessor/ffmpeg] apply FFmpegFixupM3u8PP only for videos with aac codec()
Remita Amine fc86d4eed0 [mangomolo] fix typo
Remita Amine 7d273a387a [mangomolo] add support for Mangomolo embeds
Remita Amine 6ad0219556 [common] add helper method for Wowza Streaming Engine format extraction
Remita Amine 98b7506e96 [toutv] add support for authentication(closes )
Sergey M․ 52dc8a9b3f
[franceinter] Fix upload date extraction
Sergey M․ 9d8985a165
[tv4] Fix hls and hds formats (Closes )
Sergey M․ f5e008d134
release 2016.09.15
stepshal 490b755769
Improve some id regexes
Sergey M․ 1dec2c8a0e
[adobepass] Change mvpd cache section name
In order to better emphasize it's relation to Adobe Pass
Sergey M․ dcce092e0a
[extractor/common] Simplify _get_netrc_login_info and carry long lines
Sergey M․ 32443dd346
[extractor/common] Update _get_login_info's comment
Sergey M․ 2133565cec
[extractor/common] Simplify _get_login_info
Sergey M․ 1da50aa34e
[YoutubeDL] Improve Adobe Pass options' wording
Sergey M․ d2522b86ac
[options] Actually print Adobe Pass options sections in --help
Sergey M․ 537f753399
[options] Improve Adobe Pass wording
Sergey M․ c849836854
[utils] Improve _hidden_inputs
Sergey M․ eb5b1fc021
[crunchyroll] Fix authentication (Closes )
Sergey M․ 95be29e1c6
[twitch] Fix api calls (Closes , closes )
Remita Amine c035dba19e [bellmedia] add support for more sites
Remita Amine 87148bb711 [adobepass] rename --ap-mso-list option to --ap-list-mso
Remita Amine 797c636bcb [ap] improve adobe pass names and parse error handling
Sergey M․ 0002962f3f
[franceinter] Improve extraction (Closes )
Sergey M․ 3e4185c396
[utils] Use native french month names
Sergey M․ f6717dec8a
[utils] Improve month_by_name and add tests
renalid a942d6cb48
[utils,franceinter] Add french months' names and fix extraction
Update of the "FranceInter" radio extractor : webpages HTML structure
had changed, the extractor didn't work. So I updated this extractor to
get the mp3 URL and all details.
Yen Chi Hsuan 961516bfd1
[kwuo:song] Improve error detection (closes )
Yen Chi Hsuan 6db354a9f4
[kuwo] Update _TESTS
Remita Amine 353f340e11 [go] fix typo
Remita Amine 014b7e6b25 [go] add support for free full episodes()
stepshal 925194022c
Improve some _VALID_URLs
Sergey M․ b690ea15eb
[viafree] Fix test
Remita Amine 5712c0f426 [adobepass] remove unnecessary option
Yen Chi Hsuan 86d68f906e
[bilibili] Fix extraction for videos without backup_url ()
Yen Chi Hsuan 4875ff6847
[bilibili] Remove copyrighted test cases
I can't find any English or Chinese material that claims BiliBili has
bought legal redistribution permissions for copyrighted products from
copyrighted holders.

References for removed test cases:
"刀语": https://en.wikipedia.org/wiki/Katanagatari, by White Fox
"哆啦A梦": https://en.wikipedia.org/wiki/Doraemon, by Shin-Ei Animation
"岳父岳母真难当": https://en.wikipedia.org/wiki/Serial_(Bad)_Weddings, by Les films du 24
"混沌武士": https://en.wikipedia.org/wiki/Samurai_Champloo, by Manglobe

I shouldn't have added them to _TESTS
Remita Amine 1b6712ab23 [adobepass] add specific options for adobe pass authentication
- add --ap-username and --ap-password option to specify
TV provider username and password in the cmd line
- add --ap-retries option to limit the number of retries
- add --list-ap-msi-ids to list the supported TV Providers
Sergey M․ 8414c2da31
[adobepass] PEP 8
Sergey M․ 45396dd2ed
[nhk] Fix extraction (Closes )
Remita Amine 7a7309219c [adobepass] add an option to specify mso_id and support for ROGERS TV Provider(closes )
Sergey M․ a6ccc3e518
[safari] Improve ids regexes ()
Sergey M․ 1d16035bb4
[kaltura] Improve audio detection
Sergey M․ e8bcd982cc
[kaltura] Skip chun format
Sergey M․ a5ff05df1a
[extractor/generic] Add vimeo embed that requires Referer passed
Sergey M․ d002e91986
[vimeo:ondemand] Pass Referer along with embed URL ()
Yen Chi Hsuan be45730226
[nbc] Add new extractor for NBC Olympics (, )
Sergey M․ ee7e672eb0
[tube8] Remove proxy settings from test
Sergey M․ 0307d6fba6
release 2016.09.11.1
Sergey M․ 1c81476cbb
release 2016.09.11
Sergey M․ bc9186c882
[tvplay] Remove unused import