Commit Graph

8675 Commits (25042f73722c37e4ec88030cf69e23ae76c4359b)

Author SHA1 Message Date
Remita Amine 9fa5789279 [viceland] fix info extraction(closes )
Remita Amine d2ac04674d [viceland] Add new extractor()
Remita Amine 1fd6e30988 [adobepass] create separate class for adobe pass authentication
Sergey M․ 884cdb6cd9
[life:embed] Improve extraction
Remita Amine 9771b1f901 [theplatform] use _get_netrc_login_info and fix session expiration check()
Remita Amine 2118fdd1a9 [common] add separate method for getting netrc ligin info
Sergey M․ 320d597c21
[vgtv] Detect geo restricted videos ()
Remita Amine aaf44a2f47 [uplynk] Add new extractor
phi 097eba019d bug fix for extractor xiami.py
Before applying this patch, when downloading resources from xiami.com, it crashes with these:
Traceback (most recent call last):
  File "/home/phi/.local/bin/youtube-dl", line 11, in <module>
    sys.exit(main())
  File "/home/phi/.local/lib/python3.5/site-packages/youtube_dl/__init__.py", line 433, in main
    _real_main(argv)
  File "/home/phi/.local/lib/python3.5/site-packages/youtube_dl/__init__.py", line 423, in _real_main
    retcode = ydl.download(all_urls)
  File "/home/phi/.local/lib/python3.5/site-packages/youtube_dl/YoutubeDL.py", line 1786, in download
    url, force_generic_extractor=self.params.get('force_generic_extractor', False))
  File "/home/phi/.local/lib/python3.5/site-packages/youtube_dl/YoutubeDL.py", line 691, in extract_info
    ie_result = ie.extract(url)
  File "/home/phi/.local/lib/python3.5/site-packages/youtube_dl/extractor/common.py", line 347, in extract
    return self._real_extract(url)
  File "/home/phi/.local/lib/python3.5/site-packages/youtube_dl/extractor/xiami.py", line 116, in _real_extract
    return self._extract_tracks(self._match_id(url))[0]
  File "/home/phi/.local/lib/python3.5/site-packages/youtube_dl/extractor/xiami.py", line 43, in _extract_tracks
    '%s/%s%s' % (self._API_BASE_URL, item_id, '/type/%s' % typ if typ else ''), item_id)
  File "/home/phi/.local/lib/python3.5/site-packages/youtube_dl/extractor/common.py", line 562, in _download_json
    json_string, video_id, transform_source=transform_source, fatal=fatal)
  File "/home/phi/.local/lib/python3.5/site-packages/youtube_dl/extractor/common.py", line 568, in _parse_json
    return json.loads(json_string)
  File "/usr/lib/python3.5/json/__init__.py", line 312, in loads
    s.__class__.__name__))
TypeError: the JSON object must be str, not 'NoneType'

This patch solves exactly this problem.
Sergey M․ 5ec5461e1a
[pbs] Clarify comment on http formats
Sergey M․ 542130a5d9
[pbs] Fix description extraction and update tests
Sergey M․ 82997dad57
[franceculture] Fix extraction (Closes )
Sergey M․ 647a7bf5e8
[pornotube] Fix extraction (Closes )
Sergey M․ 77afa008dd
[4tube] Fix metadata extraction (Closes )
Yen Chi Hsuan db535435b3
[bigflix] Remove an invalid test
There's no video anymore
Sergey M․ c2a453b461
[imgur] Fix width and height extraction (Closes )
Sergey M․ cd29eaab95
[vbox7] Remove unused imports
Sergey M․ e97c55ee6a
[expotv] Improve extraction and update test
Sergey M․ cb55908e51
[vbox7] Fix extraction (Closes )
Yen Chi Hsuan e581224843
[tapely] Remove extractor. It's shut down
Closes 
Remita Amine f50365e91c [pbs] add test for videos with undocumented http formats and remove unused import
Sergey M․ c366f8d30a
[24video] Add support for me and xxx TLDs
Sergey M․ 6a26c5f9d5
[muenchentv] Fix extraction (Closes )
Sergey M․ bd6fb007de
[24video] Fix comment count extraction
Sergey M․ b69b2ff736
[sunporno] Add support for embed URLs
Sergey M․ 794e5dcd7e
[sunporno] Fix metadata extraction (Closes )
Remita Amine f0d3669437 [hgtv] Add new extractor(closes )
Remita Amine 3cddb8d6a7 [pbs] check all http formats and remove unnecessary request
- some of the quality that not reported in the documentation
are available(4500k, 6500k)
- the videoInfo request doesn't work for a long time
Sergey M․ 990d533ee4
[crunchyroll] Add support for HLS (Closes )
Sergey M․ a3be69b7f0
[viu] Remove from extractors
Sergey M․ 0fd1b1624c
[goldenmoustache] Remove extractor (Closes )
Now uses dailymotion
Sergey M․ 367976d49f
[drtuber] Improve title extraction
Sergey M․ 0aef0771f8
[drtuber] Make dislike count optional (Closes )
Sergey M․ 0c070681c5
[chirbit] Fix extraction (Closes )
Sergey M․ 30b25d382d
[francetvinfo] Relax _VALID_URL
Sergey M․ 7f2ed47595
[rtlnl] Relax _VALID_URL (Closes )
Sergey M․ c3fa77bdef
[formula1] Relax _VALID_URL (Closes )
Remita Amine 57ce8a6d08 [wat] improve extraction()
add alternative method to extract http formats
works even if the video is geo-restricted or removed
from public access(most of the cases)
Yen Chi Hsuan 69d8eeeec5
[ctsnews] Fix extraction
Remita Amine 20ef4123b9 [uol] remove unused import
Remita Amine 4e62d26aa2 [uol] Add new extractor()
Sergey M․ 3a380766d1
[rbmaradio] Improve, simplify and extract all formats (Closes )
Petr Zvoníček affaea0688
[rbmaradio] Fixed extractor
Sergey M․ 77426a087b
[sonyliv] Improve (Closes )
Sukhbir Singh 8991844ea2
[sonyliv] Add new extractor
Sergey M․ 082395d0a0
[extractor/generic] Add proper default to _search_json_ld call
Sergey M․ e8ed7354e6
[flipagram] Add proper default to _search_json_ld call
Sergey M․ 1e7f602e2a
[condenast] Make _search_json_ld call non fatal
Sergey M․ 522f6c066d
[bbc] Add proper default to _search_json_ld call
Sergey M․ 321b5e082a
[extractor/common] Respect default in _search_json_ld
Sergey M․ 3711fa1eb2
Revert "[flipagram] Make _search_json_ld non fatal"
This reverts commit d34995a9e3.
Sergey M․ 395c74615c
Revert "[extractor/generic] Make _search_json_ld non fatal"
This reverts commit 958849275f.
Yen Chi Hsuan 3dc240e8c6
[sohu] Update _TESTS (closes )
Yen Chi Hsuan a41a6c5094
[chaturbate] Skip the invalid test
Yen Chi Hsuan d71207121d
[biqle] Skip an invalid test
Yen Chi Hsuan b1c6f21c74
[aparat] Fix extraction
Yen Chi Hsuan 412abb8760
[bilibili] Update _TESTS
Yen Chi Hsuan f17d5f6d14
[features.aol.com] Fix _TESTS
Remita Amine 6bb801cfaf [cwtv] extract http formats
Sergey M․ de02d1f4e9
[rozhlas] Fix regexes and improve extraction (Closes )
Petr Zvoníček e1f93a0a76
[rozhlas] Add new extractor
Yen Chi Hsuan b2bd968f4b
[kuwo:singer] Fix extraction
Sergey M․ d92cb46305
[discoverygo] Add extractor (Closes )
Sergey M․ d34995a9e3
[flipagram] Make _search_json_ld non fatal
Sergey M․ 958849275f
[extractor/generic] Make _search_json_ld non fatal
Sergey M․ 998f094452
[bbc] Remove proxy from test
Sergey M․ aaa42cf0cf
[bbc] PEP 8
Sergey M․ 9fb64c04cd
[bbc] Add support for morph embeds (Closes )
Remita Amine f9622868e7 [bbc] preserve format_id backward compatibility
Remita Amine 37768f9242 [common] correctly lower the preference of m3u8 master manifest format
Sergey M․ a1aadd09a4
[tnaflixnetworkbase] Improve title extraction
Sergey M․ b47a75017b
[tnaflix] Fix metadata extraction (Closes )
Remita Amine e37b54b140 [fox] fix theplatform release url query
Yen Chi Hsuan c1decda58c
[openload] Fix extraction (closes )
Remita Amine ad152e2d95 [bbc] fix test
Remita Amine b0af12154e [bbc] reduce requests and improve format_id
Remita Amine d16b3c6677 [common] extract partOfTVSeries info in json-ld
Remita Amine c57244cdb1 [common] lower the preference of m3u8 master manifest format
Remita Amine a7e5f27412 [bbc] improve extraction
- extract f4m and dash formats
- improve format sorting and listing
- improve extraction of articles with `otherSettings.playlist`
Remita Amine 089a40955c [pokemon] improve _VALID_URL
Remita Amine d73ebac100 [pokemon] Add new extractor(closes )
Remita Amine e563c0d73b [condenast] fallback to loader.js if video.js fail
Sergey M․ 8122e79fef
[gamekings] Remove remnants
Sergey M․ fe3ad1d456
[adultswim] Remove superfluous md5 from test
Sergey M․ 038a5e1a65
[adultswim] Add support for trailers (Closes )
Sergey M․ 84bc23b41b
[archiveorg] PEP 8
Sergey M․ 46933a15d6
[extractor/common] Support root JSON-LD lists (Closes )
Sergey M․ 3859ebeee6
[tvplay] Capture and output native error message
Remita Amine d50aca41f8 [archiveorg] improve format extraction(closes )
Remita Amine 0ca057b965 [jwplatform] add support for playlist extraction and relative urls and improve audio detection
Sergey M․ 5ca968d0a6
[tvplay] Extract series metadata
Sergey M․ f0d31c624e
[tvplay] Add support for subtitles (Closes )
Remita Amine 08c655906c [5min] fix _VALID_URL(closes )
Remita Amine 5a993e1692 [natgeo] fix tests(closes )
Remita Amine a7d2953073 [extractors] add tvp:embed import
Remita Amine fdd0b8f8e0 [tvp] extract video id from the webpage(fixes )
Remita Amine f65dc41b72 [naver] extract upload date
Yen Chi Hsuan 962250f7ea
[cbslocal] Fix timestamp parsing (closes )
Remita Amine b02b960c6b [naver] improve extraction(closes )
Remita Amine 4f427c4be8 [condenast] improve extraction
Sergey M․ 8a00ea567b
[natgeo:episodeguide] Do not shadow url from outer scope
Remita Amine 8895be01fc [5min] fix _VALID_URL
Remita Amine 52e7fcfeb7 [engadget] Relax _VALID_URL
Remita Amine 2396062c74 [5min] delegate extraction to AolIE
recently the 5min SenseHandler request return
HTTP Error 503: Service Unavailable error
Remita Amine 14704aeff6 [kaltura] remove debugging line
Remita Amine 3c2c3af059 [extractors] change imports for national geographic extractors
Remita Amine 1891ea2d76 [nationalgeographic] Add support for National Geographic Episode Guide
Remita Amine 1094074c04 [kaltura] extract subtitles and reduce requests
Remita Amine 217d5ae013 [vodplatform] Add new extractor
Remita Amine 8b40854529 [common] lower proto_preference of rtsp formats
Most of the time the RtspFD fail to download videos but it report
success of the download with this output:
[mpv] 0 bytes
[download] 100% of 0.00B
Sergey M․ 8d3b226b83
[gamekings] Remove extractor
Now covered by generic jwplayer
Remita Amine 42b7a5afe0 [limelight] extract http formats
Sergey M․ 9cb0e65d7e
[ntvru] Fix extraction
Sergey M․ b070564efb
[extractor/common] Support multiple properties in _og_search_property
Philipp Hagemeister 6a9b3b61ea [comedycentral] Re-add shortnames
In cc99d4f826, the shortname feature got deleted by accident. Re-add it as a separate IE.
Sergey M․ e03d3e6453
[cwtv] Add support for cwtvpr.com (Closes )
Remita Amine a70e45f80a [limelight] keep videos marked as previewStream
e382b953f0 (commitcomment-18472915)
Sergey M․ 697655a7c0
[safari] Relax url regexes (Closes )
Remita Amine e382b953f0 [limelight] skip preview and drm protected videos
Yen Chi Hsuan 116e7e0d04
[bloomberg] Support BPlayer() players (closes )
Sergey M․ cf03e34ad3
[yandexmusic:track] Fix extraction (Closes )
Sergey M․ fa9f1d16b8
[dailymotion:playlist] Carry long line
Dave 485fedf6fd
[dailymotion:playlist] Optimize download archive processing
Jaime Marquínez Ferrándiz da0baba5c8 [rtve] Fix extraction for some videos
For example http://www.rtve.es/alacarta/videos/documentos-tv/documentos-tv-descredito/3574098/.
Jaime Marquínez Ferrándiz bb9f3bfedf Revert "[rtve] Fix extraction ()"
This reverts commit c39b2ed990.

Apparently outside of Spain using 'auth/resources' is required ().
Sergey M․ dbc0b39b91
[tv2] Improve extraction
Sergey M․ 481c5c5137
[tv2:article] Fix extraction (Closes )
Sergey M․ 0cacae2807
[twitch:clips] Sort formats
Sergey M․ 74ba450a81
[twitch:clips] Fix extraction (Closes )
Sergey M․ db19df6ca0
[extractor/generic] Add test for
Sergey M․ fbdf8d15d1
[soundcloud] Add _extract_urls ()
Sergey M․ 94aae01548
[extractor/generic] Extract all soundcloud embeds (Closes )
Sergey M․ 39eef54cf0
[ard:mediathek] Skip unavailable test
Sergey M․ 05c8268c81
[shared] Modernize and make more robust
Sergey M․ 289a16b4f3
[shared] Respect redirect URL (Closes )
Sergey M․ 40090e8d51
[extractor/common] Improve is_suitable
In order to fix breakage introduced by a3aa814b77
Sergey M․ 3e050d51d4
[orf:oe1] Relax _VALID_URL
Sergey M․ ced70c8640
[cbc] PEP 8
Sergey M․ 9a700deea4
[instagram] Remove duplicate field in test
Sergey M․ dc35ba0eba
[mgtv] Fix typo
Sergey M․ 88bd486b9a
[cbc] Improve extraction for videos embedded with clipId
Sergey M․ 7f8b92e3cf
[bigflix] Update tests
Yen Chi Hsuan 35f6e0ff36
[mtv.de] Skip 2 geo-restricted tests
Yen Chi Hsuan 326fa4e6e5
[generic] Skip an invalid test
Yen Chi Hsuan c74299a72c
[cmt] Detect unavailable videos and update _TESTS
Yen Chi Hsuan 10a1bb3a78
[mtv] Fix for videos with missing bitrates
Yen Chi Hsuan 4d3e543c73
Update extractors.py
Yen Chi Hsuan 05d1e7aaa9
[generic] Fix an MTV test and another test that breaks nosetests
Yen Chi Hsuan a3aa814b77
Update _TESTS for MTV sites
Yen Chi Hsuan 5c32a77cad
[nextmovie] Remove extractor
This domain name now redirects to mtv.com
Yen Chi Hsuan cc99d4f826
[comedycentral] Remove IEs for *.cc.com except tosh.cc.com
All other subdomains now redirects to cc.com/* URLs
Yen Chi Hsuan 712c7530ff
[mtv] Extract more metadata and more
1. Remove MTVIggyIE. All www.mtviggy.com URLs now redirects to
   www.mtv.com
2. Fix MTVDEIE
3. Return multiple URLs from _transform_rtmp_url. This is for
   tosh.cc.com
Sergey M․ 0a147785e8
[camdemy] Extract duration properly
Sergey M․ 59eaf69e33
[camdemy] Fix camdemy
Sergey M․ e8be2943a7
[smotri] Modernize, make more robust and fix tests
Sergey M․ 9513c1eb17
[tvp] Update dash format comment
Sergey M․ ae6fff4e64
[onet] Enable dash formats
Sergey M․ 5a65668e25
[dcn] Enable dash formats
Sergey M․ f75e6890db
[telegraaf] Make hls non fatal
Sergey M․ d9cb92c840
[telegraaf] Enable dash formats
Sergey M․ 94c04a3c79
[arkena] Enable dash formats
Sergey M․ f094834857
[extractor/common] Add support for $ in SegmentTemplate in MPD manifests
Déstin Reed 111de00289
[DailyMail] Improve title and description extraction
Sergey M․ b4a131e1a5
[facebook] Relax _VALID_URL (Closes )
Sergey M․ f1991ce928
[arkena] Skip dash formats
Sergey M․ 3a8947650b
[arkenaplay] Remove extractor
Sergey M․ 1979969f91
[extractor/generic] Add support for arkena embeds
Sergey M․ 0673741af3
[extractors] Add imports for arkena and lcp
Sergey M․ c8e170b209
[lcp] Improve extraction
Sergey M․ bbe1f3634a
[arkena] Improve extraction (Closes )
Rob van Bekkum 4671dd41b2
[arkena:lcp] Add extractors
Sergey M․ b13647cf3c
[eporner] Fix extraction (Closes )
Sergey M․ add7d2a0e2
[pornhub] Make error regex less ambiguous (Closes )
Sergey M․ e298d3a08c
[youtube] Fix authentication (Closes )
Sergey M․ fd8c8c7dcd
[youtube:shared] Relax _VALID_URL
Sergey M․ 9158af16cc
[bbc.co.uk:iplayer:playlist] Add support for group URLs
Sergey M․ c6668e4ad1
[bbc.co.uk:iplayer:playlist] Skip unavailable test
Sergey M․ 84e8cca48b
[youjizz] Relax _VALID_URL (Closes )
Sergey M․ 790b06b7d4
[odatv] Improve (Closes )
skacurt 740d7c49c2
[odatv] Add extractor
Sergey M․ 4e51ec5f57
[extractors] Add import for comedycentral.tv
Sergey M․ 05087d1b4c
[bbc] Improve extraction from sxml playlists
Sergey M․ a66a73ee90
[ard] Add test for rbb-online
Sergey M․ e8882e7043
[spike] Relax _VALID_URL and improve extraction (Closes )
Sergey M․ 1056821799
[viki] Fix tests (Closes )
Sergey M․ 890e6d3309
[viki] Lower m3u8 preference
http URLs are always provde the same or better quality
Sergey M․ 246080d378
[viki] Override m3u8 formats acodec
Sergey M․ b1ea680270
Revert "[bbc] extract more and better qulities from Unified Streaming Platform m3u8 manifests"
This reverts commit 0385aa6199.
Sergey M․ 45550d1039
[comedycentraltv] Add extractor (Closes )
Sergey M․ 7cdfc4c90f
[mtvservices] Strip description
Sergey M․ af21f56f98
[ard] Add support for rbb-online (Closes )
Sergey M․ 1a8f0773b6
[streamable] Fix title extraction and improve (Closes )
Zach Bruggeman 59cc5bd8bf
[streamable] Add extractor
Sergey M․ 49bc16b95e
[nintendo] Improve playlist extraction (Closes )
TRox1972 a2f9ca1e67
[nintendo] Add extractor
Sergey M․ 371ddb14fe
[extractor/generic] Change twitter:player embeds priority to lowest (Closes )
Yen Chi Hsuan 998895dffa
[cloudy] Drop videoraj.to
videoraj.ch is now a shoe-selling website, and videoraj.to domain name
is gone.
Yen Chi Hsuan aadd3ce21f
[cliphunter] Update _TESTS
Yen Chi Hsuan ae7b846203
[cbsnews] Update _TESTS of CBSNewsLiveVideoIE
Yen Chi Hsuan 21ba7d0981
[cbc] Skip geo-restricted test case
Remita Amine 224db034ab [syfy] fix extraction(closes )(closes )(closes )
Sergey M․ ad27649be3
[3qsdn] Restrict src JS regex
Sergey M․ 84571be645
[orf:tvthek] Remove test md5
Nehal Patel 7b0d333a7e
Fix unit tests for m3u8 and RTSP extractors that require ffmpeg or mplayer
Remita Amine 342f0c3682 [ninenow] correct test url
Remita Amine 38e0f16a94 [ninenow] Add new extractor(closes )
Remita Amine e910fe2fe4 [brightcove] skip ism manifests
Jaime Marquínez Ferrándiz 233b58dec7 Add extractor for rtve.es/television (fixes )
Jaime Marquínez Ferrándiz c39b2ed990 [rtve] Fix extraction ()
For http://www.rtve.es/alacarta/videos/documentos-tv/documentos-tv-revolucion-del-movil/3069778/ using 'auth/resources' fails, and other URLs seem to work fine.
Remita Amine 35ec86689c [bbc] extract only the original Unified Streaming Platform m3u8 manifests
0385aa6199 (commitcomment-18233275)
manifests with higher birate require more time to check formats
Sergey M․ a0560d8ab8
[ellentv] Improve extraction (Closes )
Remita Amine 0385aa6199 [bbc] extract more and better qulities from Unified Streaming Platform m3u8 manifests
Remita Amine 00f4764cb7 [common] extract vbr, abr and fps for Unified Streaming Platform m3u8 manifests
Sergey M․ 51c2cd0b83
[extractors] Add vk:wallpost extractor import
Sergey M․ 5f5a9d6158
[vk] Improve login
Sergey M․ 2d19fb5072
[vk:wallpost] Add extractor
Remita Amine 41aa44259d [shahid] try to bypass geo restriction and extract more metadata(closes )
Sergey M․ 7f29cf545a
[youtube] Add YouTube Red paid video reference test ()
Remita Amine 7d1219f3e0 [tmz] delegate extraction to KalturaIE
Remita Amine f1b4af7d79 [beightcove:new] remove html tags from description
Remita Amine 8a8590a617 [dbtv] delegate extraction to BrightcoveNewIE
Remita Amine 4a7a5e41f7 [tvplay] improve extraction
Yen Chi Hsuan 2a49d01600
[playvid] Update _TESTS
Blocks https://travis-ci.org/rg3/youtube-dl/jobs/143809100
Yen Chi Hsuan b99af8a51c
[biobiochiletv] Fix extraction and update _TESTS
Yen Chi Hsuan 8e7020daef
[rudo] Add new extractor
Used in biobiochile.tv
Sergey M․ 5c4dcf8172
[vidzi] Add support for embed URLs (Closes )
Sergey M․ e9fb6a4bbe
[youtube] Relax TFA regexes
Yen Chi Hsuan e2dbcaa1bf
[vuclip] Fix extraction
Yen Chi Hsuan ae01850165
[miomio] Fix _TESTS
Yen Chi Hsuan c3baaedfc8
[miomio] Support new 'h5' player (closes )
Depends on 
Yen Chi Hsuan 0b68de3cc1 Merge pull request from remitamine/html5_media
[extractor/common] add helper method to extract html5 media entries
Sergey M․ 865b087224
[roosterteeth] Improve (Closes )
Nehal Patel 3121b25639
[roosterteeth] Add extractor
Sergey M․ ab52bb5137
[animeondemand] Fix typo
Sergey M․ 61a98b8623
[lynda] Remove md5 from test (Closes )
Sergey M․ 6daf34a045
[facebook] Fix typo and break when found video_data (Closes )
Yen Chi Hsuan c03adf90bd
[generic] Add the test. Closes
Yen Chi Hsuan 0ece114b7b
[vimeo] Recognize non-standard embeds ()
Remita Amine 8cc9b4016d [srmediathek] extend _VALID_URL(closes )
Remita Amine 31eeab9f41 [ard] fix f4m extraction and skip tests with 404 errors
Sergey M․ 9558dcec9c
[youtube:user] Preserve user/c path segment
Sergey M․ 6e6b70d65f
[extractor/generic] Properly comment out a test
Sergey M․ 9e4f5dc1e9
[animeondemand] Pass num for episode based videos
Sergey M․ 1f7258a367
[animeondemand] Add support for full length films (Closes )
Sergey M․ 0af985069b
[flipagram] Improve extraction (Closes )
Sergey M․ 0de168f7ed
[extractor/generic] Detect schema.org/VideoObject embeds
Sergey M․ 95b31e266b
[extractor/common] Add expected_type in json ld routines
Sergey M․ 6b3a3098b5
[extractor/common] Extract more metadata for VideoObject in _json_ld
Sergey M․ 2de624fdd5
[extractor/common] Introduce filesize metafield for thumbnails
Déstin Reed 3fee7f636c
[flipagram] Add extractor