Commit Graph

547 Commits (d563fb32ba5ef4b1a8061fca27edf3b1ad7eb8fc)

Author SHA1 Message Date
Yen Chi Hsuan 6a9cb29509
[extractor/common] Fix json dumping with --geo-bypass
The line "[debug] Using fake IP %s (%s) as X-Forwarded-For." was printed
to stdout even with -j/-J, which breaks the resultant JSON.
8 years ago
Yen Chi Hsuan ca27037171
[bilibili] Fix extraction of videos with double quotes in titles
Closes #13387
8 years ago
Sergey M․ cb1e6d8985
release 2017.06.12 8 years ago
Sergey M․ 9932ac5c58
[ChangeLog] Actualize 8 years ago
Sergey M․ 537191826f
release 2017.06.05 8 years ago
Sergey M․ 130880ba48
[ChangeLog] Actualize 8 years ago
Sergey M․ 941ea38ef5
release 2017.05.29 8 years ago
Sergey M․ 99bea8d298
[ChangeLog] Actualize 8 years ago
Yen Chi Hsuan a49eccdfa7
[youtube] Parse player_url if format URLs are encrypted or DASH MPDs are requested
Fixes #13211
8 years ago
Yen Chi Hsuan d16c0121b9
[youku] Extract more metadata (closes #10433) 8 years ago
Sergey M․ 28dbde9cc3
release 2017.05.26 8 years ago
Sergey M․ cc304ce588
[ChangeLog] Actualize 8 years ago
Yen Chi Hsuan 98a0618941
[ChangeLog] Update after the fix for #11381 8 years ago
Yen Chi Hsuan c130f0a37b
[tudou] Merge into youku extractor (fixes #12214)
Also, there are no tudou playlists anymore. All playlist URLs points to youku
playlists.
8 years ago
Yen Chi Hsuan d3d4ba7f24
[youku:show] Fix extraction 8 years ago
Yen Chi Hsuan 5552c9eb0f
[utils] Recognize more patterns in strip_jsonp()
Used in Youku Show pages
8 years ago
Yen Chi Hsuan 59ed87cbd9
[youku] Fix extraction (closes #13191) 8 years ago
Yen Chi Hsuan 5192ee17e7
[postprocessor/ffmpeg] Fix metadata filename handling on Python 2
Fixes #13182
8 years ago
Sergey M․ 2570e85167
release 2017.05.23 8 years ago
Sergey M․ 9dc5ab041f
[ChangeLog] Actualize 8 years ago
Sergey M․ bfdf6fcc66
release 2017.05.18.1 8 years ago
Sergey M․ bcaa1dd060
[ChangeLog] Actualize 8 years ago
Sergey M․ 9221d5d7a8
[ChangeLog] Fix typo 8 years ago
Sergey M․ 9d63e57d1f
release 2017.05.18 8 years ago
Sergey M․ 3bc1eea0d8
[ChangeLog] Actualize 8 years ago
Sergey M․ 6f3c632c24
release 2017.05.14 8 years ago
Sergey M․ 09b866e171
[ChangeLog] Actualize 8 years ago
Yen Chi Hsuan 116283ff64
[liveleak] Fix extraction (#12053) 8 years ago
Yen Chi Hsuan 7274f3d0e9
[douyu] Support Douyu shows (closes #12228) 8 years ago
Sergey M․ a7ed6b341c
release 2017.05.09 8 years ago
Sergey M․ cbd84b5817
[ChangeLog] Actualize 8 years ago
Sergey M․ 4ac0f573ef
release 2017.05.07 8 years ago
Sergey M․ 3892a9f4ab
[ChangeLog] Actualize 8 years ago
Yen Chi Hsuan 228cd9bb90
[bilibili] Fix video downloading (closes #13001) 8 years ago
Yen Chi Hsuan 4ac6dc3732
[vice] Support Vice articles (closes #12968) 8 years ago
Yen Chi Hsuan cc7bda4fff
[vice] Fix extraction for non en_us videos (closes #12967) 8 years ago
Yen Chi Hsuan 50ad078b7b
[gdcvault] Fix extraction for videos with gdc-player.html
Closes #12733
8 years ago
Yen Chi Hsuan 0c26548601
[cda] Implement birthday verification (closes #12789) 8 years ago
Yen Chi Hsuan 5401bea27f
[leeco] Fix extraction (closes #12974)
Seems on mobile devices a similar API is used, but I always get an AD
with mimicking that API.
8 years ago
Sergey M․ e0c1e9a98c
release 2017.05.01 8 years ago
Sergey M․ 086041e2f8
[ChangeLog] Actualize 8 years ago
Yen Chi Hsuan 24eb7c2578
[xtube] Fix extraction with non-standard JSON 'sources'
Closes #12734

Thanks @paulguy for the fix!
8 years ago
Sergey M․ bc8a2ea071
release 2017.04.28 8 years ago
Sergey M․ 7527923371
[ChangeLog] Actualize 8 years ago
Yen Chi Hsuan e31fed95b4
[youtube] Recognize new locale-based player URLs (fixes #12885) 8 years ago
Sergey M․ a82f41841d
release 2017.04.26 8 years ago
Sergey M․ 30a4ab191a
[ChangeLog] Actualize 8 years ago
Yen Chi Hsuan 01cb57016f
[iqiyi] Fix extraction of Yule videos 8 years ago
Yen Chi Hsuan fbf56be213
[iqiyi] Fix playlist detection (#12504) 8 years ago
Yen Chi Hsuan 54f54fcca7
[socks] Report errors elegantly when credentails are required but missing
In some non-standard implementations, the server may respond AUTH_USER_PASS
even if's not listed in available authentication methods. (it should
respond AUTH_NO_ACCEPTABLE per standards)
8 years ago
Yen Chi Hsuan facfd79f9a
[azubu] Remove extractor as the site is gone (closes #12813) 8 years ago
Yen Chi Hsuan 3110bb937d
[porn91] Fix extraction (closes #12814) 8 years ago
Yen Chi Hsuan 06d0ad9a4e
[brightcove] Support URLs with bcpid instead of playerID
Fixes #12482
8 years ago
Sergey M․ 16a09aefe3
release 2017.04.17 8 years ago
Sergey M․ fa7a6e6de6
[ChangeLog] Actualize 8 years ago
Sergey M․ 022250a594
release 2017.04.16 8 years ago
Sergey M․ 76b5f99617
[ChangeLog] Actualize 8 years ago
Sergey M․ d24bd4ffdb
release 2017.04.15 8 years ago
Sergey M․ b9859e2735
[ChangeLog] Actualize 8 years ago
Sergey M․ 38d2f8325f
release 2017.04.14 8 years ago
Sergey M․ 6f4dd6667b
[ChangeLog] Actualize 8 years ago
Sergey M․ 1730878167
release 2017.04.11 8 years ago
Sergey M․ 689cd458a6
[ChangeLog] Actualize 8 years ago
Sergey M․ 4372436504
release 2017.04.09 8 years ago
Sergey M․ eb8cc8ea3b
[ChangeLog] Actualize 8 years ago
Yen Chi Hsuan 4c03973296
[airmozilla] Fix extraction (closes #12670) 8 years ago
Sergey M․ b022f4f600
release 2017.04.03 8 years ago
Sergey M․ e2435ba5f3
[ChangeLog] Actualize 8 years ago
Sergey M․ b56e41a701
release 2017.04.02 8 years ago
Sergey M․ a76c25146a
[ChangeLog] Actualize 8 years ago
Sergey M․ 9e691da067
release 2017.03.26 8 years ago
Sergey M․ 82eefd0be0
[ChangeLog] Actualize 8 years ago
Yen Chi Hsuan f7923a4c39
[ChangeLog] Update after #12307 8 years ago
Yen Chi Hsuan 51ef4919df
[afreecatv] Fix extraction (closes #12179) 8 years ago
Sergey M․ a3ccd6bd11
release 2017.03.24 8 years ago
Sergey M․ 7963b6cba8
[ChangeLog] Actualize 8 years ago
Sergey M․ 093dad9e25
release 2017.03.22 8 years ago
Sergey M․ e8686e51d7
[ChangeLog] Actualize 8 years ago
Sergey M․ 0e9a73e612
release 2017.03.20 8 years ago
Sergey M․ 0ecdd3adbd
[ChangeLog] Actualize 8 years ago
Yen Chi Hsuan 68220649fa
[ChangeLog] Update after #12099 8 years ago
Yen Chi Hsuan ea883a687c
[openload] Fix extraction (closes #10408)
Thanks to @makgun02

Ref: http://pastebin.com/raw/JX9gHFUz
8 years ago
Sergey M․ 7d539ee10a
release 2017.03.16 8 years ago
Sergey M․ 6ad476079d
[ChangeLog] Actualize 8 years ago
Sergey M․ 5db83d79bf
release 2017.03.15 8 years ago
Sergey M․ 2a751e137f
[ChangeLog] Actualize 8 years ago
Yen Chi Hsuan 4605c94d1a
[__init__] Fix missing subtitles if --add-metadata is used (#12423)
The previous fix for #5594 is incorrect
8 years ago
Sergey M․ a8e687a4da
release 2017.03.10 8 years ago
Sergey M․ f9e5c92c94
[ChangeLog] Actualize 8 years ago
Yen Chi Hsuan 0f6b87d067
[miomio] Fix extraction
Closes #12291
Closes #12388
Closes #12402
8 years ago
Sergey M․ dccd0ab35d
release 2017.03.07 8 years ago
Sergey M․ 80146dcc6c
[ChangeLog] Actualize 8 years ago
Yen Chi Hsuan 54a3a8827b
[__init__] Metadata should be added after conversion
Fixes #5594
8 years ago
Yen Chi Hsuan 92cb5763f4
[ChangeLog] Update after #12357 8 years ago
Sergey M․ 1664702626
release 2017.03.06 8 years ago
Sergey M․ 3f116b189b
[ChangeLog] Actualize 8 years ago
Sergey M․ afa4597618
release 2017.03.05 8 years ago
Sergey M․ 75027364ba
[ChangeLog] Actualize 8 years ago
Yen Chi Hsuan 6f4e4132d8
[douyutv] Switch to the PC API to escape the 5-min limitation
Thanks @spacemeowx2 for the algo.

Ref: https://gist.github.com/spacemeowx2/629b1d131bd7e240a7d28742048e80fc

Closes #12316
8 years ago
Sergey M․ 250eea6821
release 2017.03.02 8 years ago
Sergey M․ 28d15b73f8
[ChangeLog] Actualize 8 years ago
Yen Chi Hsuan 11bb6ad1a5
[facebook] Fix extraction (closes #12323)
Almost all videos now use the pagelet type 'permalink_video_pagelet'
8 years ago
Sergey M․ 1dc24093f8
release 2017.02.28 8 years ago
Sergey M․ 11bae9cdde
[ChangeLog] Actualize 8 years ago
Yen Chi Hsuan 9bd05b5a18 [daisuki] Add new extractor (closes #4738) 8 years ago
Yen Chi Hsuan f4c68ba372
[douyu] Fix extraction and update _TESTS
They've switched from flv to hls

Closes #12301
8 years ago
Sergey M․ ef48a1175d
release 2017.02.27 8 years ago
Sergey M․ c6184bcf7b
[ChangeLog] Actualize 8 years ago
Yen Chi Hsuan 892b47ab6c
[scivee] Remove extractor (#9315)
The Wikipedia page is changed from active to down:
https://en.wikipedia.org/w/index.php?title=SciVee&diff=prev&oldid=723161154

Some other interesting bits:

$ nslookup www.scivee.tv
Server:         8.8.8.8
Address:        8.8.8.8#53

Non-authoritative answer:
www.scivee.tv   canonical name = scivee.rcsb.org.
Name:   scivee.rcsb.org
Address: 132.249.231.211

$ nslookup rcsb.org
Server:         8.8.8.8
Address:        8.8.8.8#53

Non-authoritative answer:
Name:   rcsb.org
Address: 132.249.231.77

Both IPs are from UCSD. I guess it's maintained by a lab and they don't
maintain it anymore.
8 years ago
Yen Chi Hsuan a5cf17989b
[MDR] Relax _VALID_URL and playerURL matching and update _TESTS
Ref: #12169
8 years ago
Yen Chi Hsuan 9d0c08a02c
[vevo] Fix videos with the new streams/streamsV3 format (closes #11719) 8 years ago
Yen Chi Hsuan db182c63fb
[njpwworld] Add new extractor (closes #11561) 8 years ago
Sergey M․ 204efc8509
release 2017.02.24.1 8 years ago
Sergey M․ 5d3a51e1b9
[ChangeLog] Actualize 8 years ago
Sergey M․ 6b097cff27
release 2017.02.24 8 years ago
Sergey M․ f2f7961820
[ChangeLog] Actualize 8 years ago
Sergey M․ 345b24538b
release 2017.02.22 8 years ago
Sergey M․ 63a29b6118
[ChangeLog] Actualize 8 years ago
Yen Chi Hsuan b5869560a4
[crunchyroll] Fix descriptions with double quotes (closes #12124) 8 years ago
Sergey M․ 8c6c88c7da
release 2017.02.21 8 years ago
Sergey M․ 159aaaa9d0
[ChangeLog] Actualize 8 years ago
Sergey M․ 28e35f5070
release 2017.02.17 8 years ago
Sergey M․ cf3704c132
[ChangeLog] Actualize 8 years ago
Yen Chi Hsuan fef51645d6
[theplatform] Recognize URLs with whitespaces (closes #12044) 8 years ago
Yen Chi Hsuan a4a554a793
[generic] Try parsing JWPlayer embedded videos (closes #12030) 8 years ago
Sergey M․ 2480b056c1
release 2017.02.16 8 years ago
Sergey M․ 3aa25395aa
[ChangeLog] Actualize 8 years ago
Sergey M․ 58a65ba852
release 2017.02.14 8 years ago
Sergey M․ cedf08ff54
[ChangeLog] Actualize 8 years ago
Yen Chi Hsuan f8514630db
[einthusan] Fix extraction (closes #11416)
The old test URLs are no longer valid, so I replace them with the one
from #11416
8 years ago
Yen Chi Hsuan 8d38dafbbf
ChangeLog: update after #12085 8 years ago
Sergey M․ 9b92a5917b
release 2017.02.11 8 years ago
Sergey M․ 3e2274c8b7
[ChangeLog] Actualize 8 years ago
Sergey M․ 55d4de2283
release 2017.02.10 8 years ago
Sergey M․ 61ee556aea
[ChangeLog] Actualize 8 years ago
Sergey M․ 013877298d
release 2017.02.07 8 years ago
Sergey M․ c87f95f991
[ChangeLog] Actualize 8 years ago
Yen Chi Hsuan 9c14fe9681
[gaskrank] Minor change and update ChangeLog after #11685 8 years ago
Yen Chi Hsuan 019f4c0371
[bandcamp] Fix extraction for incomplete albums
Closes #11727
8 years ago
Yen Chi Hsuan caf0f5f8b7
[iwara] Fix extraction (closes #11781) 8 years ago
Yen Chi Hsuan e4e50f60b1
[googledrive] Fix extraction on Python 3.6
Since Python 3.6, invalid escape sequences are deprecated. It's likely
that there are invalid escape sequences somewhere on the webpage, so
instead of unescaping the whole webpage, just unescape the URL.

See https://bugs.python.org/issue27364. That change was designed for
string literals, while it affects the 'unicode_escape' encoding as well.
The code path is:

str.decode('unicode_escape')
    codecs.unicode_escape_decode()
        PyUnicode_DecodeUnicodeEscape()
8 years ago
Sergey M․ a713a86755
release 2017.02.04.1 8 years ago
Sergey M․ 7bccd5fc8a
[ChangeLog] Actualize 8 years ago
Sergey M․ 3144eccf55
[ChangeLog] Actualize 8 years ago
Sergey M․ 31487eb974
release 2017.02.04 8 years ago
Yen Chi Hsuan daac118bf4
[ChangeLog] Update after #11901 8 years ago
Yen Chi Hsuan d7f9242e30
[ChangeLog] Update after #11565 8 years ago
Yen Chi Hsuan a22b2fd19b
[youtube] Fix ytsearch* when cookies are provided
Closes #11924

The API with `page` is no longer used in browsers, and YouTube always
returns {'reload': 'now'} when cookies are provided.

See http://youtube.github.io/spfjs/documentation/start/ for how SPF
works. Basically appending static link with a `spf` parameter yields the
corresponding dynamic link.
8 years ago
Yen Chi Hsuan bd8f48c78b
[bilibili] Support new Bangumi URLs (closes #11845)
To reduce complexity, I don't support old Bangumi URLs directly via
_VALID_URL. Instead, I choose to let it go to generic redirection. An
example can be found in #10190:

http://bangumi.bilibili.com/anime/v/40062
8 years ago
Jaime Marquínez Ferrándiz 020c5df52d [elpais] Fix extraction for some URLs (closes #11765) 8 years ago
Sergey M․ 5069594993
release 2017.02.01 8 years ago
Sergey M․ b996b88092
[ChangeLog] Actualize 8 years ago
Yen Chi Hsuan 000f207944
[prosiebensat1] Update ChangeLog 8 years ago
Sergey M․ d7e215b42d
release 2017.01.31 8 years ago
Sergey M․ 3a528ffd89
[ChangeLog] Actualize 8 years ago
Sergey M․ 4d2fdb07c4
release 2017.01.29 8 years ago
Sergey M․ fe323a4800
[ChangeLog] Actualize 8 years ago
Sergey M․ d41ed6d243
release 2017.01.28 8 years ago
Sergey M․ 815d2a36d8
[ChangeLog] Actualize 8 years ago
Sergey M․ 2417d41535
release 2017.01.25 8 years ago
Sergey M․ 2c302cf66b
[ChangeLog] Actualize 8 years ago
Sergey M․ c3a65c3de0
release 2017.01.24 8 years ago
Sergey M․ ee4c091ce5
[ChangeLog] Actualize 8 years ago
Yen Chi Hsuan bc35ed3fb6
[nextmedia] Add support for NextTV (壹電視) 8 years ago
Sergey M․ 9d5b29c881
release 2017.01.22 8 years ago
Sergey M․ 6c031a35f3
[ChangeLog] Actualize 8 years ago
Yen Chi Hsuan 7c20b7484c
[nextmedia] Support redirected URLs 8 years ago
Yen Chi Hsuan 4447fb2332
[cspan] Support Ustream embedded videos
Closes #11547
8 years ago
Sergey M․ 1560baacc6
release 2017.01.18 8 years ago
Sergey M․ 460f61fac4
[ChangeLog] Actualize 8 years ago
Sergey M․ c1c2fe2045
release 2017.01.16 8 years ago
Sergey M․ ddd53c392e
[ChangeLog] Actualize 8 years ago
Yen Chi Hsuan 16e2c8f771
[brightcove] Recognize another player ID
Closes #11688
8 years ago
Yen Chi Hsuan dcae7b3fdc
[niconico] Allow login via cookies
Some codes are borrowed from #7968, which is by @jlhg

Closes #7968
8 years ago
Sergey M․ 5d4c7daa49
release 2017.01.14 8 years ago
Sergey M․ 0b94510cd0
[ChangeLog] Actualize 8 years ago
Sergey M․ 31ea2ad89d
release 2017.01.10 8 years ago
Sergey M․ 2184d44361
[ChangeLog] Actualize 8 years ago
Philipp Hagemeister bfedb2cc5a small fix to Changelog format 8 years ago
Philipp Hagemeister 8084951b7f [egghead:course] Add support for egghead.io course playlists
Individual egghead videos are already handled by the generic/Wistia extractors.
8 years ago
Sergey M․ e7ea724cb9
release 2017.01.08 8 years ago
Sergey M․ e60166020b
[ChangeLog] Actualize 8 years ago
Yen Chi Hsuan 1219201143
[ChangeLog] Update after #11581
[ci skip]
8 years ago
Yen Chi Hsuan 24d8a75982
[discoverygo] Fix JSON data parsing
HTMLParser, which is used by extract_attributes, already unescapes
attribute values with HTMLParser.unescape. They shouldn't be unescaped
again, to there may be parsing errors.

Ref: #11219, #11522
8 years ago
Sergey M․ 7232bb299b
release 2017.01.05 8 years ago
Sergey M․ 2b12e34076
[ChangeLog] Actualize 8 years ago
Sergey M․ 2021b650dd
release 2017.01.02 8 years ago
Sergey M․ b890caaf21
[ChangeLog] Actualize 8 years ago
Sergey M․ dc48a35404
release 2016.12.31 8 years ago
Sergey M․ 1ea0b727c4
[ChangeLog] Actualize 8 years ago
Sergey M․ 5e77c0b58e
release 2016.12.22 8 years ago
Remita Amine ab3091feda
[ChangeLog] Actualize 8 years ago
Sergey M․ 90352a8041
release 2016.12.20 8 years ago
Sergey M․ 1f6a79b0af
[ChangeLog] Actualize 8 years ago
Yen Chi Hsuan 8ab7e6c4cc
[kaltura] Improve widget ID extraction (closes #11480) 8 years ago
Sergey M․ f73d7d5074
release 2016.12.18 8 years ago
Sergey M․ 52a1d48d9f
[ChangeLog] Actualize 8 years ago
Sergey M․ 38be3bc568
release 2016.12.15 8 years ago
Sergey M․ d7ef47bffd
[ChangeLog] Actualize 8 years ago
Yen Chi Hsuan 5c32a5be95 [openload] Recognize oload.tv URLs (#10408) 8 years ago