Commit Graph

10197 Commits (b969d12490a4c618ee65b5731084db4c95209af8)

Author SHA1 Message Date
Sergey M․ efcba804f6 [udemy] Extract formats from view_html (Closes #8979) 9 years ago
Sergey M․ 6dee688e6d [youtube:playlistsbase] Restrict playlist regex (Closes #8986) 9 years ago
Sergey M․ eedb7ba536 [YoutubeDL] Sort imports 9 years ago
Sergey M․ dcf77cf1a7 [YoutubeDL] Sanitize final URLs (Closes #8991) 9 years ago
Sergey M․ 17bcc626bf [utils] Extract sanitize_url routine 9 years ago
Sergey M․ b5a5bbf376 [mailru] Extend _VALID_URL (Closes #8990) 9 years ago
Yen Chi Hsuan e68d3a010f [twitter] Fix extraction (closes #8966)
HLS and DASH formats are no longer appeared in test cases. I keep them
for fear of triggering new errors.
9 years ago
Yen Chi Hsuan d10fe8358c [generic] Add a test case for brightcove embed
Closes #8862
9 years ago
Yen Chi Hsuan d6c340cae5 [brightcove] Extract more formats (#8862) 9 years ago
Yen Chi Hsuan 5964b598ff [brightcove] Support alternative BrightcoveExperience layout
The full URL lays in the `data` attribute of <object> (#8862)
9 years ago
Philipp Hagemeister 62cdb96f51 release 2016.03.26 9 years ago
Sergey M․ 6e6bc8dae5 Use urlencode_postdata across the codebase 9 years ago
Sergey M․ 15707c7e02 [compat] Add compat_urllib_parse_urlencode and eliminate encode_dict
encode_dict functionality has been improved and moved directly into compat_urllib_parse_urlencode
All occurrences of compat_urllib_parse.urlencode throughout the codebase have been replaced by compat_urllib_parse_urlencode

Closes #8974
9 years ago
Sergey M․ 2156f16ca7 [thescene] Fix extraction and improve style (Closes #8978) 9 years ago
Sergey M․ 4db441de72 [once] Relax _VALID_URL (Closes #8976) 9 years ago
Philipp Hagemeister 0be8314dc8 release 2016.03.25 9 years ago
Yen Chi Hsuan d7f62b049a [iqiyi] Update enc_key 9 years ago
Yen Chi Hsuan 3bb3356812 [douyutv] Extend _VALID_URL 9 years ago
Sergey M․ 98e68806fb [mnet] Improve (Closes #8958) 9 years ago
Kagami Hiiragi e031768666 [mnet] Add new extractor 9 years ago
Sergey M․ 5eb7db4ee9 [udemy] Add support for new URL schema 9 years ago
Sergey M․ f0e83681d9 [udemy] Extract formats from outputs 9 years ago
Sergey M․ ff9d5d0938 [udemy] Improve course enrolling 9 years ago
Sergey M․ d041a73674 [extractor/__init__] Add youtube:live and sort youtube extractors alphabetically 9 years ago
Sergey M․ f07e276a04 [youtube:live] Add extractor (Closes #8959) 9 years ago
Sergey M․ 993271da0a [nytimes] Tolerate missing metadata (Closes #8952) 9 years ago
Sergey M․ 369e7e3ff0 [iprima] Fix extraction (Closes #8953) 9 years ago
Sergey M․ 5767b4eeae [mtv] Fix description extraction (Closes #8962) 9 years ago
Yen Chi Hsuan 622d19160b [utils] Clarify Python versions affected by buggy struct module 9 years ago
Yen Chi Hsuan 32d88410eb [tumblr] Add a test with Instagram embed
Closes #8817
9 years ago
Yen Chi Hsuan 5a51775a58 [generic] Extract Instagram embeds (#8817) 9 years ago
Yen Chi Hsuan 87696e78d7 [instagram] Unescape description (#8817) 9 years ago
Yen Chi Hsuan c4096e8aea [instagram] Extract embed videos (#8817) 9 years ago
Yen Chi Hsuan fc27ea9464 [tumblr] Support Vine embeds (#8817) 9 years ago
Yen Chi Hsuan 088e1aac59 [generic] Support Vine embeds (#8817) 9 years ago
Sergey M 4333d56494 Merge pull request #8898 from dstftw/fragment-retries
Add --fragment-retries option (Fixes #8466)
9 years ago
Sergey M․ 882c699296 [tunein] Fix stream data extraction (Closes #8899, closes #8924) 9 years ago
Yen Chi Hsuan efbed08dc2 [utils] Encode hostnames before passing to urllib
With IDN (Internationalized Domain Name) and a proxy, non-ascii URLs
are passed down to urllib/urllib2, causing UnicodeEncodeError

Fixes #8890
9 years ago
Jaime Marquínez Ferrándiz 7da2c87119 Add extractor for thescene.com (closes #8929) 9 years ago
Sergey M․ c6ca11f1b3 [once] Prevent ads from embedding into m3u8 playlists (Closes #8893) 9 years ago
Sergey M․ 2beeb286e1 [laola1tv] Add support for livestreams (Closes #8934) 9 years ago
Sergey M․ cc7397b04d [ceskatelevize] Make m3u8 formats extraction non fatal (Closes #8933) 9 years ago
Sergey M․ bc5d16b302 [animeondemand] Skip dash for now 9 years ago
Sergey M․ 85c637b737 [animeondemand] Extract teaser when no full episode available (#8923) 9 years ago
Sergey M․ 5c69f7a479 [animeondemand] Respect startvideo (Closes #8923) 9 years ago
Sergey M․ ff5873b72d [motherless] Detect friends only videos 9 years ago
Sergey M․ 065c4b27bf [xhamster:embed] Extract vars (Closes #8912) 9 years ago
Sergey M․ 1600ed1ff9 [rutv] Improve flash version pattern (Closes #8911) 9 years ago
Sergey M․ 5886b38d73 Add support for https for all extractors as preventive and future-proof measure 9 years ago
Sergey M․ 0cef27ad25 Add missing r prefix for _VALID_URLs 9 years ago
Sergey M․ 12af4beb3e [mailru] Add support for https (Closes #8920) 9 years ago
Sergey M․ 9016d76f71 [YoutubeDL] Improve _format_note 9 years ago
Sergey M․ 3c5d183c19 [animeondemand] Extract all formats (Closes #8906) 9 years ago
Sergey M․ 3e8bb9a972 [animeondemand] Detect geo restriction 9 years ago
Yen Chi Hsuan daef04a4e7 [kwuo] Fix KuwoChartIE and KuwoSingerIE and accept new URL forms 9 years ago
Yen Chi Hsuan 2648918c81 [vlive] Fix creator extraction (closes #8814) 9 years ago
Yen Chi Hsuan 9e3c2f1d74 [openload] Misc improvements
* Add thumbnail
* Detect errors (#6469)
* Match more (#6469, #8489)
9 years ago
Yen Chi Hsuan 2bfeee69b9 [openload] Add new extractor (closes #8489) 9 years ago
Yen Chi Hsuan 664bcd80b9 [tudou] Use InAdvancePagedList (closes #8884) 9 years ago
Sergey M․ 3c20208eff [francetv] Improve formats extraction 9 years ago
Sergey M․ db264e3cc3 [francetvinfo] Add support for france3-regions and strip title (Closes #7673) 9 years ago
Sergey M․ 96a9f22d98 [discovery] Relax _VALID_URL (Closes #8903) 9 years ago
Sergey M․ 40025ee2a3 [postprocessort/ffmpeg] Allow embedding webvtt into webm (Closes #8874) 9 years ago
Sergey M․ 298c04b464 [91porn] Use common messages' wording 9 years ago
Sergey M․ d95114dd83 [91porn] Unquote final URL (Closes #8881) 9 years ago
Sergey M․ fa023ccb2c [biobiochiletv] Fix extraction, extract m3u8 formats and overall improve (Closes #7314) 9 years ago
jjatria e36f4aa72b [biobiotv] Add extractor 9 years ago
Sergey M․ f1ced6df51 [cda] Improve and simplify (Closes #8805) 9 years ago
Kacper Michajłow 8b0d7a66ef [cda] Add new extractor for cda.pl
Fixes #8760
9 years ago
Sergey M․ 3aec71766d [safari:api] Separate extractor (Closes #8871) 9 years ago
Sergey M․ 16a8b7986b [downloader/fragment] Document fragment_retries 9 years ago
Sergey M․ 617e58d850 [downloader/{common,fragment}] Fix total retries reporting on python 2.6 9 years ago
Sergey M․ e33baba0dd [downloader/dash] Add fragment retry capability
YouTube may often return 404 HTTP error for a fragment causing the
whole download to fail. However if the same fragment is immediately
retried with the same request data this usually succeeds (1-2 attemps
is usually enough) thus allowing to download the whole file successfully.
So, we will retry all fragments that fail with 404 HTTP error for now.
9 years ago
Sergey M․ 721f26b821 [downloader/fragment] Add report_retry_fragment 9 years ago
Sergey M․ 52bb437e41 [options] Add --fragment-retries option 9 years ago
Jaime Marquínez Ferrándiz 782b1b5bd1 [utils] lookup_unit_table: Match word boundary instead of end of string 9 years ago
Sergey M․ 0d769bcb78 [extractor/generic] Fix missing byte literal prefix 9 years ago
remitamine 4cd70099ea [hbo] Add new extractor 9 years ago
Jaime Marquínez Ferrándiz 09fc33198a utils: lookup_unit_table: Use a stricter regex
In parse_count multiple units start with the same letter, so it would match different units depending on the order they were sorted when iterating over them.
9 years ago
John Peel d5aacf9a90 Added format_id to the filers on -f. 9 years ago
Sergey M․ 19e2617a6f [commonprotocols] Add generic support for rtmp URLs (Closes #8488) 9 years ago
Sergey M․ edd9b71c2c [extractor/generic] Add a test for m3u playlist served without proper Content-Type 9 years ago
Sergey M․ 5940862d5a [extractor/generic] Detect m3u playlists served without proper Content-Type 9 years ago
Sergey M․ de6c51e88e [extractor/generic] Fix direct link semantics 9 years ago
Sergey M․ 303dcdb995 [extractor/generic] Simplify upload_date extraction 9 years ago
Sergey M․ 20938f768b [extractor/generic] Add another test for generic m3u8 9 years ago
Sergey M․ 955737b2d4 [extractor/generic] Force Content-Type to lowecase 9 years ago
Sergey M․ 263eff9537 [extractor/generic] Properly extract format id from Content-Type
Fixes extraction for cases like: audio/x-mpegURL; charset=utf-8
9 years ago
Sergey M․ cae21032ab [theplatform] Improve geo restriction detection 9 years ago
remitamine 6187091532 [once] check http formats availability 9 years ago
Philipp Hagemeister 0d33166ec5 release 2016.03.18 9 years ago
remitamine 87c03c6bd2 [theplatform] remove unnecessary import 9 years ago
remitamine 4c92fd2e83 [theplatform] always force theplatform to return a smil for _extract_theplatform_smil 9 years ago
Sergey M․ e3d17b3c07 [noz] Fix extraction on python 2.6 by means of using compat_xpath 9 years ago
Sergey M․ 810c10baa1 [utils] Use compat_xpath 9 years ago
Sergey M․ 57f7e3c62d [compat] Add compat_xpath 9 years ago
Sergey M․ 0d0e282912 [animeondemand] Fix typo and improve 9 years ago
Sergey M․ 85e8f26b82 [animeondemand] Improve extraction 9 years ago
Sergey M․ b57fecfddd [animeondemand] Add test 9 years ago
Sergey M․ 8c97e7efb6 [animeondemand] Expand episode title regex (Closes #8875) 9 years ago
Sergey M․ cc162f6a0a [crunchyroll] Fix custom _download_webpage (Closes #8883) 9 years ago
remitamine cf45ed786e [wistia] extract more metadata 9 years ago
remitamine 574b2a7393 [nbc:nbcnews] improve extraction(fixes #6922)
- extract more metadata and formats
- relax regex
9 years ago
remitamine 9f02ff537c [theplatform] extract brightcove once formats 9 years ago
remitamine 0436ec0e7a [once] Add new format extractor 9 years ago
Yen Chi Hsuan 11f12195af [youtube] Added itag 91
Seen in https://www.youtube.com/watch?v=jMN4cxyhJjk
9 years ago
remitamine a646a8cf98 [sbs] improve extraction(fixes #3811)
- extract error messages
- force the platform smil url(previously the manifest param
in the query is not respected which make theplatform return non working
mp4 files for some videos)
9 years ago
remitamine 63f41d3821 [bravotv] Add new extractor(#4657) 9 years ago
Sergey M․ c5229f3926 [utils] PEP 8 9 years ago
Sergey M․ 96f4f796fb [brightcover] Remove unused import 9 years ago
Sergey M․ 70cab344c4 [udemy] Improve course id v4 regex 9 years ago
Quan Hua a7ba57dc17 [udemy] Update course id regex to cover v4 layout (Closes #8753, closes #8868, closes #8870) 9 years ago
remitamine 83548824c2 Merge pull request #8092 from bpfoley/twitter-thumbnail
[utils] Add extract_attributes for extracting html tag attributes
9 years ago
remitamine 354dbbd880 [brightcove:new] extract protocol-less embed URLs(closes #2914) 9 years ago
remitamine 23edc49509 [tv3] Add new extractor(closes #8059) 9 years ago
remitamine 48254c3f2c [brightcove] some improvements and fixes
- use FFmpeg downloader to download m3u8 formats extracted
from BrightcoveNew(some of the m3u8 media playlists use AES-128)
- update comment and update_url_query to handle url query
9 years ago
remitamine 2cab48704c [thestar] Add new extractor(closes #5955) 9 years ago
remitamine 64d4f31d78 [brightcove:new] update embed_in_page embeds regex to match non numeric ref id 9 years ago
remitamine 0c9ff24041 [noz] fix extraction in python 2.6 9 years ago
Yen Chi Hsuan 3ff8279e80 [kuwo:mv] Fix the test and extraction of georestricted MVs 9 years ago
remitamine cb6e477dfe [aljazeera] update the extractor to use BrightcoveNewIE 9 years ago
remitamine edfd93518e [svt] extract dashhbbtv formats(#8867) 9 years ago
remitamine 89807d6a82 [brightcove] extract dash formats and detect audio formats 9 years ago
remitamine 49dea4913b Merge pull request #8513 from remitamine/dash-sort
[extractor/common] fix dash formats sorting
9 years ago
Sergey M․ dec2cae0a7 [twitch:playlistbase] Clarify pagination bug
Pagination bug has been fixed by twitch on 15.03.2016.
9 years ago
remitamine cf6cd07396 [noz] extract f4m and m3u8 formats 9 years ago
remitamine 975b9c9ab0 [brightcove:new] detect m3u8 manifests by M2TS container 9 years ago
remitamine 8ac73bdbe4 [brightcove:new] Add support for non numeric ref: preffixed video ids 9 years ago
remitamine 877f440f7b [rice] Add new extractor(closes #1736) 9 years ago
remitamine d13bdc3824 [brightcove] raise ExtractorError on 403 errors and fix regex to work with tenplay 9 years ago
remitamine 744daf9418 [gameinformer] remove unused imports 9 years ago
remitamine bf475e1990 [tlc] fix extraction and update extractor to use BrightcoveNewIE 9 years ago
remitamine 203f3d779a [gameinformer] update the extractor to use BrightcoveNewIE 9 years ago
remitamine 4230c4894d [external/downloader] fix rtmp downloading using FFmpegFD 9 years ago
Philipp Hagemeister 6bb266693f release 2016.03.14 9 years ago
remitamine 5d53c32701 [usatoday] Add new extractor(closes #8655) 9 years ago
remitamine 2e7e561c1d Merge pull request #8611 from remitamine/ffmpegfd
[downloader/external] Add FFmpegFD
9 years ago
remitamine d8515fd41c [downloader/external] pass configuration args to ffmpeg 9 years ago
remitamine 694c47b261 [external/downloader] don't pass -t and -ss to ffmpeg 9 years ago
remitamine 77dea16ac8 [downloader/external] check for ffmpeg availablity when it used for m3u8 download 9 years ago
remitamine 6ae27bed01 [download/external] move the check for multiple selected formats to get_suitable_downloader 9 years ago
remitamine da1973a038 [extractor/__init__] disable time range downloading 9 years ago
remitamine be24916a7f [downloader/rtsp] Add rtsp and mms downloader 9 years ago
remitamine 2cb99ebbd0 [downloader/external] add can_download mathod for checking downloader availibilty and support 9 years ago
remitamine 91ee320bfa [downloader/external] wrap available_opt in a list 9 years ago
remitamine 8fb754bcd0 Merge pull request #8821 from remitamine/list-thumbnails-order
[YoutubeDL] check for --list-thumbnails immediately after processing them
9 years ago
remitamine b7b72db9ad [YoutubeDL] check for --list-thumbnails immediately after processing them 9 years ago
remitamine 634415ca17 [downloader/external] skip FFmpegFD when requesting multiple formats 9 years ago
Sergey M․ 2f7ae819ac [utils] PEP 8 9 years ago
Sergey M․ 0a477f8731 [vice:show] Add extractor (Closes #8847) 9 years ago
remitamine a755f82549 [ffmpeg] convert format ext to ffmpeg output formats codes 9 years ago
Sergey M․ 7f4173ae7c [mixcloud] Fix view count extraction (Closes #8831, closes #8845) 9 years ago
Sergey M․ fb47597b09 [bbc] Generalize unit table lookup and add parse_count 9 years ago
Sergey M․ 450b233cc2 [bbc] Update test 9 years ago
Sergey M․ b7d7674f1e [bbc] Update test 9 years ago
Sergey M․ 0e832c2c97 [bbc] Improve title and description extraction (Closes #8826, closes #8822) 9 years ago
Benjamin Congdon 8e4aa7bf18 [bbc] Fix BBC Extractor to work with 'School Report' 9 years ago
remitamine a42dfa629e [makerschannel] Add new extractor(closes #8839) 9 years ago
remitamine b970dfddaf [minoto] Add new extractor 9 years ago
Sergey M․ 46a4ea8276 [safari] Remove unused imports 9 years ago
Sergey M․ 3f2f4a94aa [extractor/generic] Extract f4m formats from final URLs 9 years ago
Sergey M․ f930e0c76e [extractor/generic] Extract f4m formats and refactor common info 9 years ago
Sergey M․ 0fdbb3322b [extractor/common] Add _parse_f4m_formats routine 9 years ago
Sergey M․ e9c8999ede [safari] Fix authentication 9 years ago
Sergey M․ 73cbd709f9 [safari] Respect kaltura session (Closes #7491) 9 years ago
Sergey M․ 9dce3c095b [kaltura] Respect kaltura session 9 years ago
remitamine e5a2e17a9c [kaltura] optimize url info extraction 9 years ago
remitamine 0ec589fac3 Merge pull request #8827 from remitamine/safari
[safari] extract free and preview videos(#7491)
9 years ago
remitamine 36bb63e084 [dw] add support for article pages(closes #8790) 9 years ago
remitamine 91d6aafb48 [dw] add support for audio pages 9 years ago
remitamine c8868a9d83 [dw] Add new extractor 9 years ago
remitamine 09f572fbc0 [extractor/common] add transform_source to _download_smil and _extract_smil_formats 9 years ago
Sergey M․ 58e6d097d8 [googledrive] Relax _VALID_URL (Closes #8829) 9 years ago
remitamine 15bf934de5 Merge pull request #8819 from remitamine/simple-webpage-requests
[extractor/common] simplify using data, headers and query params with _download_* methods
9 years ago
remitamine cdfee16818 [extractor/common] add data, headers and query params to _request_webpage 9 years ago
remitamine bcb668de18 [safari] extract free and preview videos(#7491) 9 years ago
remitamine fac7e79277 [kaltura] add support for videos with reference id 9 years ago
Yen Chi Hsuan a6c8b75904 [common] Use mimeType to determine file extensions (#8766) 9 years ago
Yen Chi Hsuan 25cb05bda9 [utils] Remove codec2ext
This function is orignally used for determining file extensions of DASH
formats. Now in DASH, ext is determined by mime_type. See #8766 for more
information.
9 years ago
Sergey M․ 883c052378 [audioboom] Improve robustness and extract uploader (Closes #8812) 9 years ago
Benjamin Congdon 61f317c24c Added extractor for AudioBoom.com 9 years ago
Yen Chi Hsuan 64f08d4ff2 Merge pull request #8766 from yan12125/dash-detect-ext
Detect file extensions of DASH formats from their codecs
9 years ago
Yen Chi Hsuan e738e43358 [facebook] Support videos in groups
Viewing/Downloading videos in groups requires logging in, even for
those in public groups.

Fixes #6951.
9 years ago
Jaime Marquínez Ferrándiz f6f6217a98 [facebook] Don't override variable in list comprehension 9 years ago
Yen Chi Hsuan 31db8709bf [iqiyi] Update enc_key 9 years ago
Yen Chi Hsuan 5080cbf9fd [facebook] Handle escaped swf params
Fixes #8713
9 years ago
Yen Chi Hsuan 9880124196 [facebook] Fix for m.facebook.com URLs 9 years ago
Yen Chi Hsuan 9c7b509b2a [facebook] Merge FacebookPostIE into FacebookIE
Fixes #8713
9 years ago
Sergey M․ 5d583bdf6c [YoutubeDL] Improve _format_note 9 years ago
Sergey M․ 1e501364d5 [vimeo:ondemand] Clarify IE_NAME 9 years ago
Sergey M․ 74278def2e [vimeo:ondemand] Separate ondemand extractor (Closes #8330, closes #8801) 9 years ago
Sergey M․ e375a149e1 [livestream] Properly build smil URLs (#8794) 9 years ago
Benjamin Congdon ac45505528 Added flag for 'allow_audio_only' format in Twitch queries 9 years ago
Sergey M․ 46c329d6f6 [arte] Improve extraction (Closes #8768) 9 years ago
Sergey M․ 1818e4c2b4 [arte] Fix typo 9 years ago
Sergey M․ e7bd17373d [sexu] Improve extraction (Closes #8782) 9 years ago
aystroganov@gmail.com c58e74062f [Sexu] fix extractor 9 years ago
Yen Chi Hsuan 6d210f2090 [utils] Add more codecs to codec2ext
BBC uses avc3. Here's an example (thanks to @remitamine for this example)

http://rdmedia.bbc.co.uk/dash/ondemand/bbb/2/client_manifest-common_init.mpd

See also https://trac.ffmpeg.org/ticket/5217
9 years ago
Yen Chi Hsuan af7d5a63b2 [common] Document protocol http_dash_segments 9 years ago
Yen Chi Hsuan e41acb6364 [safari] Don't pollute std_headers (#8778) 9 years ago