Commit Graph

12715 Commits (e7db6759e4c4ca5099a997285642e60b84eee2c6)

Author SHA1 Message Date
Sergey M․ c183e14f89
[viu] Relax _VALID_URL (closes )
Sergey M․ 093dad9e25
release 2017.03.22
Sergey M․ 8e5a7c5e67
[pluralsight] Omit module title from video title (closes )
Sergey M․ e1e35d1ac6
[pornhub] Improve extraction and style (closes )
Throaway 21fbf0f955
[pornhub] Decode obfuscated video URL (closes )
John Hawkinson 97952bdb78 [generic] Add test for Senate ISVP iframe embed
John Hawkinson 8a8cc339b6 [senateisvp] Allow https URL scheme for embeds
Vijay Singh 957f453429 [Openload.co] Fixed Extraction
They did it again. just a minor change though. here's quick fix
Sergey M․ 0e9a73e612
release 2017.03.20
Sergey M․ 9487ce03e9
[YoutubeDL] Allow multiple input URLs to be used with stdout as output template
John Hawkinson 46b18f2349 [BostonGlobe] New. Nonstandard version of Brightcove.
Has a "data-brightcove-video-id" instead of a "data-video-id," otherwise
pretty much just Brightcove. Except the Globe isn't all Brightcove
videos, so fallback to Generic, too.

Also, abstract playlist_from_matches() from generic.py to common.py, and use
it here.

History of these changes can be found in
51170427d4b1143572a498dedaee61863a5b2c5b.
Remita Amine 772b5ff57f [toongoggles] Add new extractor(closes )
Sergey M․ f68ef1e2ab
[medialaan] Remove unrelated test
Sergey M․ febfe1e262
[adobepass] Detect and output error on authz token extraction ()
Vijay Singh 5f0daab1ca [openload] Fix extraction
Sergey M․ 2a721cdff2
[medialaan] Fix and improve extraction (closes )
mrBliss e7a51a4c02
[vtm] Add extractor (closes )
Implementation of the approach described in .
Remita Amine 3e5856d860 [discoverynetworks] add support for more domains and bypass geo restiction
Yen Chi Hsuan ea883a687c
[openload] Fix extraction (closes )
Thanks to @makgun02

Ref: http://pastebin.com/raw/JX9gHFUz
Sergey M․ 7d539ee10a
release 2017.03.16
Philipp Hagemeister 0efbc6b56d [options] Mention flac support and sort alphabetically among the audio formats
Philipp Hagemeister 21bfcd3d6e [postprocessor/ffmpeg] Add support for flac
Requested at http://stackoverflow.com/q/42828041/35070
Sergey M․ b51dc9db0e
[extractor/common] Extract SMIL formats from jwplayer
Sergey M․ a309684285
[extractor/generic] Add forgotten return for jwplayer formats
Remita Amine ba448445b8 [redbull] improve extraction
- extract 1080p quality
- correct ttml subtitle ext
- catch api errors
- reduce request size
Sergey M․ 5db83d79bf
release 2017.03.15
Vijay Singh 398887b4c0 [Openload] Fixed Extraction
They did changed it again.
Sergey M․ 66bf351f80
[facebook] Make title optional (closes )
Sergey M․ 9d08963022
[telecinco] Add test for
Sergey M․ e313d209c2
[mitele] Add support for ooyala videos (closes )
Vijay Singh ff9d509d20 [openload] Fix extraction
Just a minor fix for openload
Lucas M c1795ca6c8 [streamable] Update API URL
Starsam80 8c99623259 [crunchyroll] Extract season name
Sergey M․ 57b0ddb35f
[discoverygo] Actualize test
Sergey M․ a28f8d7396
[discoverygo] Bypass geo restriction
Sergey M․ 7049799470
[discoverygo:playlist] Add extractor (closes )
Yen Chi Hsuan 4605c94d1a
[__init__] Fix missing subtitles if --add-metadata is used ()
The previous fix for  is incorrect
Sergey M․ a8e687a4da
release 2017.03.10
Sergey M․ c2ee861c6d
[extractor/generic] Make title optional for jwplayer embeds (closes )
Sergey M․ bd34c32bd7
[wdr] Actualize comment
runningbits f802c48660 [wdr:maus] Fix extraction and update tests
Sergey M․ 76bee08fe7
[prosiebensat1] Improve title extraction and add test
Thomas Christlieb 2913821723 [prosiebensat1] Improve title extraction (closes )
Sergey M․ 0e7f9a9b48
[dplayit] Relax playback info URL extraction
Sergey M․ 0cf2352e85
[dplayit] Separate and rewrite extractor and bypass geo restriction (closes )
Yen Chi Hsuan 0f6b87d067
[miomio] Fix extraction
Closes 
Closes 
Closes 
Sergey M․ d7344d33b1
[telequebec] Fix description extraction and update test (closes )
denneboomyo b08cc749d6 [openload] Fix extraction
Sergey M․ b68a812ea8
[extractor/generic] Add test for brigthcove UUID-like videoPlayer
Sergey M․ 2e76bdc850
[brightcove:legacy] Relax videoPlayer validation check (closes )
Yen Chi Hsuan fe646a2f10
[twitch] PEP8
Sergey M․ dccd0ab35d
release 2017.03.07
Sergey M․ e30ccf7047
[soundcloud] Update client id (closes )
Yen Chi Hsuan 54a3a8827b
[__init__] Metadata should be added after conversion
Fixes 
denneboomyo da92da4b88 Openload fix extraction ()
* Fix extraction
Sergey M․ 1664702626
release 2017.03.06
Sergey M․ 4b5de77bdb
[utils] Process bytestrings in urljoin (closes )
Sergey M․ 96182695e4
[drtv] Add geo countries to GeoRestrictedError
Sergey M․ fc11ad3833
[drtv:live] Bypass geo restriction
Yen Chi Hsuan d2b64e04b4
[addanime] Skip an invalid test
Sergey M․ 5dd376345b
[tunepk] Add extractor (closes , closes )
Sergey M․ 1a2192cb90
[extractor/common] Pass arguments to _parse_jwplayer_formats and PEP8
Sergey M․ 0236cd0dfd
[extractor/common] Improve height extraction and extract bitrate
Sergey M․ ed0cf9b383
[extractor/common] Move jwplayer formats extraction in separate method
Sergey M․ a50862b735
[downloader/external] Add missing import and PEP8
John Hawkinson 6d0fe752bf [external:ffmpeg] In test harness, limit to 10k download size
Otherwise, if you screw up a playlist test by including a playlist
dictionary key, you'll be there for eons while it downloads all the
files before erroring out.
Sergey M․ afa4597618
release 2017.03.05
Sergey M․ 5316566edc
[twitch] Use better naming and simplify (closes )
Xiao Di Guan c64c03be35
[twitch] Add basic support for two-factor authentication
Lars Vierbergen a3ba8a7acf [vier] Add support for vijf.be
vier.be and vijf.be run on the same CMS and are property of the same company,
so the same extractor can be used for both of them.
Sergey M․ 054a587de8
[redbulltv] Improve extraction (closes , closes )
Juanjo Benages 64b7ccef3e
[redbulltv] Add extractor
Yen Chi Hsuan 6f4e4132d8
[douyutv] Switch to the PC API to escape the 5-min limitation
Thanks @spacemeowx2 for the algo.

Ref: https://gist.github.com/spacemeowx2/629b1d131bd7e240a7d28742048e80fc

Closes 
Sergey M․ eb3079b6ce
[generic] Add support for rutube embeds
Sergey M․ bc82f22879
[rutube] Relax _VALID_URL
Sergey M․ 4d058c9862
[vrak] Improve and update test (closes )
Sergey M․ d16f27ca27
[brightcove:new] Add ability to smuggle geo_countries into URL
Olivier Bilodeau cbb127568a
[vrak] Add extractor
Sergey M․ d02d4fa0a9
[brightcove:new] Raise GeoRestrictedError
Sergey M․ 692fa200ca
[go] Relax _VALID_URL (closes )
Sergey M․ 9bae185ba6
[24video] Use original host for requests (closes )
Sergey M․ 4d345bf17b
[ruutu] Disable DASH formats (closes )
Due to causing out of sync issue
Sergey M․ 250eea6821
release 2017.03.02
Yen Chi Hsuan 11bb6ad1a5
[facebook] Fix extraction (closes )
Almost all videos now use the pagelet type 'permalink_video_pagelet'
Yen Chi Hsuan c9612c0487
[youtube] Mark errors about rental videos as expected
Closes 
Yen Chi Hsuan af5049f128
[adobepass] Add Charter Spectrum ()
Thanks @tv21 for the fix!
Sergey M․ 158af5242e
[utils] Carry long doc string
Sergey M․ 40df485f55
[YoutubeDL] Don't sanitize identifiers (closes )
Sergey M․ 4b8a984c67
[npo] Add support for audio
Sergey M․ 83e8fce628
[npo] Improve extraction and update tests
Sergey M․ aa9cc2ecbf
[npo] Adapt to app.php API (closes )
Sergey M․ 1dc24093f8
release 2017.02.28
Sergey M․ 43b38424a9
[azmedien:showplaylist] Improve (closes )
Alex Seiler 948519b35d
[azmedien:showplaylist] Add support for all episodes playlists
Sergey M․ 87dadd456a
[youtube:playlist] Recognize another playlist pattern (closes , closes )
Yen Chi Hsuan 7c4aa6fd6f
[daisuki] Add subtitles ()
Yen Chi Hsuan 9bd05b5a18 [daisuki] Add new extractor (closes )
Yen Chi Hsuan 0a5445ddbe [utils] Add bytes_to_long() and long_to_bytes()
Used in daisuki.net ()

Both are adapted from public domain PyCrypto:
https://github.com/dlitz/pycrypto/blob/master/lib/Crypto/Util/number.py
Yen Chi Hsuan f48409c7ac [utils] Add pkcs1pad
Used in daisuki.net ()
Yen Chi Hsuan c9619f0a17 [aes] Add aes_cbc_encrypt
Used in daisuki.net ()
Yen Chi Hsuan f4c68ba372
[douyu] Fix extraction and update _TESTS
They've switched from flv to hls

Closes 
Sergey M․ ef48a1175d
release 2017.02.27
Sergey M․ 18abb74376
[npo] Relax _VALID_URL for zapp.nl
Sergey M․ dbc01fdb6f
[hetklokhuis] Fix IE_NAME
Sergey M․ f264c62334
[npo] Add support for zapp.nl
Sergey M․ 0dc5a86a32
[npo] Add support for hetklokhuis.nl (closes )
Sergey M․ 0e879f432a
[youtube:channel] Remove duplicate test
Yen Chi Hsuan 892b47ab6c
[scivee] Remove extractor ()
The Wikipedia page is changed from active to down:
https://en.wikipedia.org/w/index.php?title=SciVee&diff=prev&oldid=723161154

Some other interesting bits:

$ nslookup www.scivee.tv
Server:         8.8.8.8
Address:        8.8.8.8#53

Non-authoritative answer:
www.scivee.tv   canonical name = scivee.rcsb.org.
Name:   scivee.rcsb.org
Address: 132.249.231.211

$ nslookup rcsb.org
Server:         8.8.8.8
Address:        8.8.8.8#53

Non-authoritative answer:
Name:   rcsb.org
Address: 132.249.231.77

Both IPs are from UCSD. I guess it's maintained by a lab and they don't
maintain it anymore.
Alex Seiler fdeea72611 [cda] Decode URL (fixes )
xbe 7fd4655256 [crunchyroll] Extract uploader name that's not a link
Provide the Crunchyroll extractor with the ability to extract uploader
names that aren't links. Add a test for this new functionality.
This fixes .
Sergey M․ fd5c4aab59
[youtube] Raise GeoRestrictedError
Sergey M․ 8878789f11
[dailymotion] Raise GeoRestrictedError
Yen Chi Hsuan a5cf17989b
[MDR] Relax _VALID_URL and playerURL matching and update _TESTS
Ref: 
Sergey M․ b3aec47665
[tvigle] Raise GeoRestrictedError
Yen Chi Hsuan 9d0c08a02c
[vevo] Fix videos with the new streams/streamsV3 format (closes )
Sergey M․ e498758b9c
[freshlive] Fix issues and improve (closes )
Ricardo Constantino 5fc8d89361
[freshlive] Add extractor
Pratyush Singh d374d943f3
[downloader/common] Limit displaying 2 digits after decimal point in sleep interval message
Sergey M․ 103f8c8d36
[xhamster] Capture and output videoClosed error ()
Sergey M․ 922ab7840b
[etonline] Add extractor (closes )
Sergey M․ 831217291a
[compat] Use try except for compat_numeric_types
Yen Chi Hsuan db182c63fb
[njpwworld] Add new extractor (closes )
Yen Chi Hsuan eeb0a95684
[extractor/common] Add 'preference' to _parse_html5_media_entries
Some websites, like NJPWorld, put different qualities on different
player pages.
Sergey M․ 231bcd0b6b
[amcnetworks] Relax _VALID_URL ()
Sergey M․ 204efc8509
release 2017.02.24.1
Sergey M․ ad3033037c
[noco] Modernize
Sergey M․ f3bc281239
[noco] Swtich login URL to https (closes )
Sergey M․ 441d7a32e5
[thescene] Extract more metadata
Thomas Christlieb 51ed496307 [thescene] Fix extraction (closes )
Remita Amine 68f17a9c2d [tubitv] use geo bypass mechanism
Remita Amine 39e7277ed1 [openload] fix extraction(closes )
Sergey M․ 42dcdbe11c
[ivi] Raise GeoRestrictedError
Sergey M․ 6b097cff27
release 2017.02.24
Sergey M․ be5df5ee31
Suppress help for all deprecated options and print warning when used
Sergey M․ f2980fddeb
[lynda:course] Add webpage extraction fallback (closes )
Sergey M․ 0f57447de7
[postprocessor/ffmpeg] Add mising space (closes )
Remita Amine 8e1409fd80 [go] sign all uplynk urls and use geo bypass only for free videos(closes )(closes )
Sergey M․ fafc2bf5a9
[options] Deprecate --autonumber-size
Sergey M․ b3175982c3
[YoutubeDL] Add more numeric fields for NA substitution in outtmpl
Sergey M․ 89db639dfe
[YoutubeDL] Rewrite outtmpl for playlist_index and autonumber for backward compatibility
Sergey M․ d0d9ade486 [YoutubeDL] Add support for string formatting operations in output template
Sergey M․ 28572a1a0b [compat] Add compat_numeric_types
Sergey M․ d5fd9a3be3
[skylinewebcams] Add extractor (closes )
Sergey M․ ada77fa544
[instagram] Add support for multi video posts (closes )
Remita Amine 9e03aa75c7 [crunchyroll] extract playlist entries ids
Remita Amine 30eaa3a702 [mgtv] fix extraction
Remita Amine c59f703610 [sohu] raise GeoRestrictedError
Remita Amine bc61c80c14 [leeco] raise GeoRestrictedError and use geo bypass mechanism
Sergey M․ 345b24538b
release 2017.02.22
Yen Chi Hsuan b5869560a4
[crunchyroll] Fix descriptions with double quotes (closes )
Sergey M․ 527ef85fe9
[dailymotion] Make comment count optional (closes )
Not served anymore
Sergey M․ 58ad6995cd
[vidzi] Add test for
Tobias Florek a86e416088 [vidzi] Add support for vidzi.cc
Sergey M․ 71e9577b94
[24video] Add support for 24video.tube (closes )
Remita Amine abd29a2ced [crackle] use geo bypass mechanism
Remita Amine 31615ac279 [viewster] use geo verifcation headers
Remita Amine fc320a40d9 Revert "[cbc] use geo bypass mechanism"
This reverts commit 86466a8b6f.
Remita Amine 7345d6d465 [tfo] Improve geo restriction detection and use geo bypass mechanism
Remita Amine 86466a8b6f [cbc] use geo bypass mechanism
Remita Amine 33dc173cdc [telequebec] use geo bypass mechanism
Remita Amine 3444844b04 [limelight] extract PlaylistService errors
Sergey M․ 8c6c88c7da
release 2017.02.21
Sergey M․ eea0716cae
[extractor/common] Print origin country for fake IP
Sergey M․ 336a76551b
[extractor/common] Do not quit _initialize_geo_bypass on empty countries
Sergey M․ dc0a869e5e
[extractor/common] Fix typo
Sergey M․ e39b5d4ab8
[extractor/common] Allow calling _initialize_geo_bypass from extractors ()
Remita Amine e469ab2528 [ninecninemedia] use geo bypass mechanism
Remita Amine 890d44b005 [adobepass] add support for Time Warner Cable(closes )
Sergey M․ 6926304472
[spankbang] Make uploader optional (closes )
Sergey M․ 3ccdde8cb7
[extractor/common] Emphasize geo bypass APIs are experimental
Sergey M․ da42ff0668
[iprima] Improve geo restriction detection and disable geo bypass
Sergey M․ 82f662182b
[iprima] Modernize
Sergey M․ 2cc7fcd338
[commonmistakes] Disable UnicodeBOM extractor test for python 3.2
Sergey M․ 6d4c259765
[svt] PEP 8
Sergey M․ c78dd35491
[nrk] PEP 8
Sergey M․ 8ffb8e63fe
[prosiebensat1] Throw ExtractionError on unsupported page type (closes )
Sergey M․ 983e9b7746
[nrk] Update _API_HOST and relax _VALID_URL
Sergey M․ c58b7ffef4
[tv4] Bypass geo restriction and improve detection
Sergey M․ f1a78ee4ef
[tv4] Switch to hls3 protocol (closes )
Sergey M․ de64e23c56
[downloader/ism] Honor HTTP headers when downloading fragments
Sergey M․ 553f6dbac7
[downloader/dash] Honor HTTP headers when downloading fragments
For example, https://www.oppetarkiv.se/video/1196142/natten-ar-dagens-mor
Sergey M․ 0aa10994f4 [options] Move geo restriction related options to separate section
Sergey M․ 4248dad92b Improve geo bypass mechanism
* Rename options to preffixly match with --geo-verification-proxy
* Introduce _GEO_COUNTRIES for extractors
* Implement faking IP right away for sites with known geo restriction
Sergey M․ 0a840f584c Rename bypass geo restriction options
Sergey M․ 0016b84e16 Add faked X-Forwarded-For to formats' HTTP headers
Sergey M․ 18a0defab0 [utils] Make random_ipv4 return unicode string
Sergey M․ 5d3fbf77d9 [viki] Improve geo restriction detection
Sergey M․ 80b59020e0 [vgtv] Improve geo restriction detection
Sergey M․ 71631862f4 [srgssr] Improve geo restriction detection
Sergey M․ 89cc7fe770 [vbox7] Improve geo restriction detection and use geo bypass mechanism
Sergey M․ 04d906eae3 [svt] Improve geo restriction detection and use geo bypass mechanism
Sergey M․ 8ab8066cf0 [pbs] Improve geo restriction detection and use geo bypass mechanism
Sergey M․ 01b1aa9ff4 [ondemandkorea] Improve geo restriction detection and use geo bypass mechanism
Sergey M․ ff4007891f [nrk] Improve geo restriction detection and use geo bypass mechanism
Sergey M․ 28200e654b [itv] Improve geo restriction detection and use geo bypass mechanism
Sergey M․ e633f21a96 [go] Improve geo restriction detection and use geo bypass mechanism
Sergey M․ d392005a79 [dramafever] Improve geo restriction detection and use geo bypass mechanism
Sergey M․ 773f291dcb Add experimental geo restriction bypass mechanism
Based on faking X-Forwarded-For HTTP header
Pierre Mdawar bf5b9d859a [utils] Introduce YoutubeDLError base class for all youtube-dl exceptions
Remita Amine 049a0f4d6d [brightcove:legacy] restrict videoPlayer value(closes )
Sergey M․ ac33accd96
[options] Mention quoted string literals for --match-filter
Sergey M․ e84888b432
[tvn24] Improve extraction (closes )
Jakub Wilk 02d9b82a23
[tvn24] Add extractor
Sergey M․ a2e3286676
[thisav] Add support for html5 media (closes )
Sergey M․ f75caf059e
[metacafe] Improve (closes )
Alex Monk bdabbc220c
[metacafe] Bypass family filter
If you don't send this user=ffilter: false cookie, it will 301 redirect you to a page asking about it, and then the title check will fail.
Remita Amine 70bcc444a9 [viceland] improve info extraction and update test
Sergey M․ 28e35f5070
release 2017.02.17
Sergey M․ 2c1f442c2b
[options] Add missing spaces
Sergey M․ bad4ccdb5d
[heise] Improve (closes )
Tobias Gruetzmacher db76c30c6e
[heise] Support videos embedded in any article.
Déstin Reed c2bde5d081
[ellentv] Improve
Sergey M․ 90fad0e74c
[openload] Fix extraction (closes )
Vijay Singh d94badc755
[openload] Semifix extraction (closes )
just updated the code. i don't do much python still i tried to convert my code. lemme know if there is any prob with it
Yen Chi Hsuan fef51645d6
[theplatform] Recognize URLs with whitespaces (closes )
Sergey M․ 4cead6a614
[einthusan] Relax _VALID_URL (closes , closes )
Yen Chi Hsuan a4a554a793
[generic] Try parsing JWPlayer embedded videos (closes )
Sergey M․ b898f0a173
[elpais] Fix typo and improve extraction (closes )
Sergey M․ 2480b056c1
release 2017.02.16
Sergey M․ eafaeb226a
[ceskatelevize] Lower priority for audio description sources ()
Sergey M․ de4d378c0c
[ceskatelevize] Prefix format ids
Sergey M․ db13c16ef8
[utils] Add support for quoted string literals in --match-filter (closes , closes , closes )
Remita Amine 1bd05345ea [amcnetworks] fix extraction(closes )
Sergey M․ 3021cf83b7
[pinkbike] Fix uploader extraction (closes )
Sergey M․ 04a741232f
[onetpl] Add support for businessinsider.com.pl and plejada.pl
Sergey M․ 43a3d9edfc
[onetpl] Add support for onet.pl (closes )
Sergey M․ d31aa74fdb
[onetmvp] Add shortcut extractor
Sergey M․ 6092ccd058
[vodpl] Make more robust and add another test (closes )
Marek Rusinowski 22ce9ad2bd
[vod.pl] Add new extractor
Sergey M․ 9a372f14b4
[pornhub] Extract video URL from tv platform site (, )
Sergey M․ 5cb2d36c82
[ceskatelevize] Extract DASH formats (closes , closes )
Ondřej Caletka fcca0d53a8
[ceskatelevize] Quick fix to revert to using old HLS-based playlist
This fixes recent changes in iVysilani. Proper patch should migrate to
MPEG-DASH version, which is now the default.
Sergey M․ 58a65ba852
release 2017.02.14
Sergey M․ 50de3dbad3
[zdf] Fix extraction (closes )
Sergey M․ 085f169ffe
[xtube] Fix extraction for both kinds of video id (closes )
Vobe f6d6ca1db3
[xtube] Improve title extraction
Sergey M․ 6e5956e6ba
[lemonde] Fallback delegate extraction to generic extractor (closes , closes )
Remita Amine 89c6691f9d [bellmedia] accept longer video id(closes )
Remita Amine 454e5cdb17 [limelight] add support referer protected videos
Remita Amine 9dad941853 [disney] improve extraction
- add support for more urls
- detect expired videos
- skip Adobe Flash Access protected videos

closes 
closes 
closes 
closes 
Remita Amine 0dac7cbb09 [hotstar] improve extraction(closes )
- extract all qualities
- detect drm protected videos
- extract more metadata
Yen Chi Hsuan f8514630db
[einthusan] Fix extraction (closes )
The old test URLs are no longer valid, so I replace them with the one
from 
Aniruddh-J 459818e280 [aenetworks] Add support for lifetimemovieclub.com
Sergey M․ 6310acf512
[youtube] Fix parsing codecs (closes )
Yen Chi Hsuan f3915452de Merge pull request from wiiaboo/python2
utils.py: Workaround TypeError with Python 2.7.13 in Windows
Ricardo Constantino 2f49bcd690
utils.py: Workaround TypeError with Python 2.7.13 in Windows
Fixes 

Tested with Windows Python 2.7.12 and 2.7.13.
Yen Chi Hsuan 68c22c4c15
[iqiyi] Update _TESTS
Sergey M․ 9b92a5917b
release 2017.02.11
Sergey M․ 3d7e3aaa0e
[pluralsight:course] Fix extraction (closes )
Sergey M․ 624c4b92ff
[facebook] Add coding cookie