Commit Graph

549 Commits (7cb51b5daf07a6a627a9084394636c570194cc4e)

Author SHA1 Message Date
Sergey M․ 78593e294c
Add references for
Sergey M․ 603fc4e0ea
[extractor/common] Add durations for DASH fragments with bare SegmentURLs
Petr Novak 41bf647e89
[extractor/common] Add support for DASH manifests with SegmentLists with bare SegmentURLs
Sergey M․ f610dbb05f
[extractor/common] Use final URL when dumping request (closes )
Remita Amine ea2295842f [common] skip Apple FairPlay m3u8 manifests(closes )
Sergey M․ 187ee66c94
[extractor/common] Add protocol for f4m formats
Sergey M․ 48107c198b
[f4m] Prefer baseURL for relative URLs (closes )
Sergey M․ 044eeb1455
[extractor/common] Respect URL query in _extract_wowza_formats (closes )
Sergey M․ 9211e3319e
[extractor/common] Prefix format id for audio only HLS formats
Remita Amine 50d808f5c9 [common] add support for jwplayer youtube embeds
M.K c110944fa2 [extractor/common] Fix typo in _parse_mpd_formats
Yen Chi Hsuan 4ed2d7b7d1 Fix flake8 issues after
Yen Chi Hsuan a88d461dff Merge pull request from Tithen-Firion/openload-phantomjs-method
Openload phantomjs method
Sergey M․ 1ed4549942
[extractor/common] Extract format id from label attribute of source tag for HTML5 videos ()
Sergey M․ dd121cc1ca
[extractor/common] Extract height from res attribute of source tag for HTML5 videos (closes )
Sergey M․ e01c3d2ef7
[extractor/common] Introduce _parse_xml
Sergey M․ b359e977b9
[extractor/common] Make HLS and DASH extraction non fatal in _parse_html5_media_entries (closes )
Sergey M․ 4850478543
[extractor/common] Add support for float durations in _parse_mpd_formats (closes )
Sergey M․ 868f79db41
[extractor/common] Fix _media_formats
Sergey M․ ac8491fcca
[extractor/common] Make _family_friendly_search optional
Sergey M․ 82889d4ae5
[extractor/common] Respect source's type attribute for HTML5 media (closes )
Sergey M․ 1141e9104b
Use relative paths for DASH fragments (closes )
10x reduced JSON size
refs 
Sergey M․ 749ca5eced
[extractor/common] Fix playlist_from_matches
Sergey M․ 4328ddf82b
[extractor/common] Add support for AMP tags in _parse_html5_media_entries
Sergey M․ c69701c6ab
[extractor/common] Improve _json_ld
Sergey M․ 96a2daa1ee
[extractor/common] Improve jwplayer subtitles extraction
Yen Chi Hsuan 6a9cb29509
[extractor/common] Fix json dumping with --geo-bypass
The line "[debug] Using fake IP %s (%s) as X-Forwarded-For." was printed
to stdout even with -j/-J, which breaks the resultant JSON.
Sergey M․ 0a268c6e11
[extractor/common] Improve jwplayer formats extraction (closes )
Sergey M․ 1afd0b0da7
[extractor/common] Return unicode string from _match_id
Sergey M․ f2e2f0c777
[extractor/common] Fix rtmp and rtsp formats' URLs in _extract_wowza_formats
Sergey M․ 6f76679804
[extractor/common] Add support for schemeless URLs in _extract_wowza_formats (closes , closes )
Sergey M․ 76d5a36391
[extractor/common] Respect Width and Height attributes in ISM manifests
Remita Amine ff6f9a6704 [extractor/common] fix typo in _extract_akamai_formats
Tithen-Firion c89267d31a Merge branch 'master' into openload-phantomjs-method
remitamine 55949fede6 [common] introduce chapters field
Sergey M․ 33a81c2c6f
[extractor/common] Extract view count from JSON-LD
Sergey M․ c89b49f743
[extractor/common] Add manifest_url for explicit group rendition formats
Sergey M․ ff99fe529e
Don't list master m3u8 playlists in format list (closes )
Sergey M․ ac9c69ace7
[extractor/common] Improve jwplayer regex
Tithen-Firion 40e41780f1 [phantomjs] add cookie support
Sergey M․ 3019cb0c99
[extractor/common] Rephrase comment
Sergey M․ ddd258f922
[test_InfoExtractor] Add m3u8 parsing test for NAME attribute in EXT-X-STREAM-INF tag
Sergey M․ 9c99bef704
[extractor/common] Use float for scaled tbr
Sergey M․ cb2520802d
[extractor/common] Improve m3u8 extraction (closes )
* Extract m3u8 parsing to separate method
* Improve rendition groups extraction
* Build stream name according stream GROUP-ID
* Ignore reference to AUDIO group without URI when stream has no CODECS
+ Add test coverage for parsing m3u8 from , ,  and twitch vod
Sergey M․ bae1404893
[extractor/common] Add support for video of WebPage context in _json_ld (closes )
Remita Amine bf1b87cd91 [common] Relax JWPlayer regex and remove duplicate urls()
Remita Amine 40fcba5edb improve coding style
Sergey M․ fd47550885
[extractor/common] Add coding cookie
Sergey M․ 4457823dda
[extractor/common] Move censorship checks to a separate method and add check for just another ISP
Random User 4f06c1c9fc Merge branch 'master' of github.com-rndusr:rg3/youtube-dl into fix/str-item-assignment
Random User c73e330e7a _find_jwplayer_data() returns dict or None
This simplifies code for callers of `_find_jwplayer_data()` which no longer have
to run `_parse_json()` on the return value.

It also makes sure that `_find_jwplayer_data()` returns either a `dict` or
`None` and nothing else.
John Hawkinson 46b18f2349 [BostonGlobe] New. Nonstandard version of Brightcove.
Has a "data-brightcove-video-id" instead of a "data-video-id," otherwise
pretty much just Brightcove. Except the Globe isn't all Brightcove
videos, so fallback to Generic, too.

Also, abstract playlist_from_matches() from generic.py to common.py, and use
it here.

History of these changes can be found in
51170427d4b1143572a498dedaee61863a5b2c5b.
Sergey M․ b51dc9db0e
[extractor/common] Extract SMIL formats from jwplayer
Sergey M․ 1a2192cb90
[extractor/common] Pass arguments to _parse_jwplayer_formats and PEP8
Sergey M․ 0236cd0dfd
[extractor/common] Improve height extraction and extract bitrate
Sergey M․ ed0cf9b383
[extractor/common] Move jwplayer formats extraction in separate method
Yen Chi Hsuan eeb0a95684
[extractor/common] Add 'preference' to _parse_html5_media_entries
Some websites, like NJPWorld, put different qualities on different
player pages.
Sergey M․ eea0716cae
[extractor/common] Print origin country for fake IP
Sergey M․ 336a76551b
[extractor/common] Do not quit _initialize_geo_bypass on empty countries
Sergey M․ dc0a869e5e
[extractor/common] Fix typo
Sergey M․ e39b5d4ab8
[extractor/common] Allow calling _initialize_geo_bypass from extractors ()
Sergey M․ 3ccdde8cb7
[extractor/common] Emphasize geo bypass APIs are experimental
Sergey M․ 4248dad92b Improve geo bypass mechanism
* Rename options to preffixly match with --geo-verification-proxy
* Introduce _GEO_COUNTRIES for extractors
* Implement faking IP right away for sites with known geo restriction
Sergey M․ 0a840f584c Rename bypass geo restriction options
Sergey M․ 0016b84e16 Add faked X-Forwarded-For to formats' HTTP headers
Sergey M․ 773f291dcb Add experimental geo restriction bypass mechanism
Based on faking X-Forwarded-For HTTP header
Yen Chi Hsuan a4a554a793
[generic] Try parsing JWPlayer embedded videos (closes )
Remita Amine 08a00eef79 [extractor/common] skip m3u8 manifests protected with Adobe Flash Access
Sergey M․ 242a14a1f6
[extractor/common] Fix audio only with audio group in m3u8 (closes )
Sergey M․ 2aec7256ae
[extractor/common] Speed-up media tags regex (closes )
Mattias Wadman 45024183ae [infoq] Add audio only format if available ()
* [infoq] Add audio only format if available

Refactor cookie code into a function.
Renamed formats to http_video, http_audio, rtmp_video
Renamed extract functions to video instead of videos as they return
one or no video.

* [infoq] Rename to _extract_cookies as it more than one

* [infoq] Remove redundant determine_ext

* [infoq] Add comment about hardcoded URL

* [infoq] Use _hidden_inputs instead of messy regex

* [infoq] Probe if audio URL is valid

Make it possible to pass headers to _is_valid_url

* [infoq] Add audio only test
Sergey M․ e228616c6e
[extractor/common] Fix initialization template (closes , closes )
Sergey M․ c58c2d63cb
[extractor/common] Document forgotten fragment base and path interfaces
Sergey M․ d04621daf4
[extractor/common] Fix duration per dash segment (closes )
Remita Amine c4251b9aaa [common] add possibility to customize akamai manifest host
Yen Chi Hsuan 4606c34e19 [extractor/common] Allow non-lang in subtitles' keys
See 264e77c406
Remita Amine a07588369f [common] improve detection for video only formats and m3u8 manifest(fixes )
Remita Amine 7fe1592073 [common] fix dash codec information for mixed videos and fragment url construction()
Sergey M․ 87a449c1ed
[extractor/common] Recognize DASH formats in html5 media entries
Remita Amine 8821a718cf [common] recognize hls manifests that contain video only formats()
Sergey M․ 4ea4c0bb22
[extractor/common] Fix Bandwidth substitution in media template (closes )
Sergey M․ f076d7972c
[extractor/common] Improve thumbnail extraction from JSON-LD
Sergey M․ c4c9b8440c
[extractor/common] Tolerate malformed RESOLUTION attribute in m3u8 manifests (closes )
Sergey M․ 3365ea8929
[extractor/common] Remove unused code
Sergey M․ 1616f9b452
[extractor/common] Fix typo
Sergey M․ 02dc0a36b7
[utils] Introduce base_url
Remita Amine b2758123c5 add Basic support for Smooth Streaming protocol()
Sergey M․ b82c33dd67
[extractor/common] Improve mpd base URL extraction (closes , closes )
Remita Amine 0384932e3d [extractor/common] try to extract non smil wowza mpd manifests
Remita Amine edd6074cea [extractor/common] detect f4m audio only formats
Sergey M․ 8d76bdf12b
[extractor/common] Mention podcast in series fields section
Yen Chi Hsuan cea364f70c
[extractor/common] Support HTML media elements without child nodes
Yen Chi Hsuan 98763ee354
[extractor/common] Add id and title helpers for generic IEs
Remita Amine e71a450956 [common] add hdcore sign to akamai f4m formats
Yen Chi Hsuan 5968d7d2fe
[extractor/common] Improved support for HTML5 subtitles
Ref: 

In a strict sense, <track>s with kind=captions are not subtitles. [1]
openload misuses this attribute, and I guess there will be more
examples, so I add it to common.py.

Also allow extracting information for subtitles-only <video> or <audio>
tags, which is the case of openload.

[1] https://www.w3.org/TR/html5/embedded-content-0.html#attr-track-kind
Sergey M․ 30d0b549be
[extractor/common] Add manifest_url for hls and hds formats
Sergey M․ 86f4d14f81
Refactor fragments interface and dash segments downloader
- Eliminate segment_urls and initialization_url
+ Introduce manifest_url (manifest may contain unfragmented data in this case url will be used for direct media URL and manifest_url for manifest itself correspondingly)
* Rewrite dashsegments downloader to use fragments data
* Improve generic mpd extraction
Sergey M․ b4c1d6e800
[extractor/common] Expose fragments interface for dashsegments formats
Sergey M․ a0d5077c8d
[extractor/common] Introduce fragments interface
Remita Amine 6ad0219556 [common] add helper method for Wowza Streaming Engine format extraction