Commit Graph

254 Commits (e70dc1d14b18447363f10c2fc11c1eea66e11a35)

Author SHA1 Message Date
Jaime Marquínez Ferrándiz f459d17018 [youtube] Add an extractor for downloading the watch history (closes )
Philipp Hagemeister 66ec019240 [youtube] do not use variable name twice
Philipp Hagemeister 7012b23c94 Match --download-archive during playlist processing (Fixes )
Philipp Hagemeister 3f8ced5144 Merge remote-tracking branch 'jaimeMF/yt-playlists'
Philipp Hagemeister cb7dfeeac4 [youtube] only allow domain name to be upper-case ()
Philipp Hagemeister 83aa529330 Support protocol-independent URLs ()
Philipp Hagemeister fccd377198 Suppor embed-only videos (Fixes )
Philipp Hagemeister 1d699755e0 [youtube] Add view_count (Fixes )
Jaime Marquínez Ferrándiz b9643eed7c [youtube:channel] Fix the extraction of autogenerated channels
The ajax pages are empty, now it looks directly in the channel's /videos page
Philipp Hagemeister 0e145dd541 Merge branch 'master' of github.com:rg3/youtube-dl
Philipp Hagemeister 9f9be844fc [youtube] Fix protocol-independent URLs (Fixes )
Jaime Marquínez Ferrándiz ca715127a2 Don't assume the 'subtitlesformat' is set in the params dict (fixes )
Jaime Marquínez Ferrándiz 880e1c529d [youtube:playlist] Login into youtube if requested (fixes )
Allows to download private playlists
Jaime Marquínez Ferrándiz dcbb45803f [youtube:playlist] Don't use the gdata api (closes )
Parse the playlist pages instead
Jaime Marquínez Ferrándiz 231516b6c9 Merge pull request from iemejia/master
[ted] support for subtitles
Jaime Marquínez Ferrándiz 4f045eef8f [youtube:channel] Fix the extraction
The page don't include the 'load more' button anymore, now we directly get the 'c4_browse_ajax' pages.
Philipp Hagemeister 08fb86c49b [youtube] Add description for YoutubeSearchDateIE ()
Philipp Hagemeister 3633d77c0f Merge remote-tracking branch 'CBGoodBuddy/ytsearchtime'
Jaime Marquínez Ferrándiz 9103bbc5cd Add the 'webpage_url' field to info_dict
The url for the video page, it must allow to reproduce the result.
It's automatically set by YoutubeDL if it's missing.
Craig Markwardt a3dd924871 Add YoutubeSearchDateIE extractor to youtube.py & __init__.py, which searches by publication date.
Jaime Marquínez Ferrándiz cf51923545 [youtube] Remove vevo test
The video is no longer available and it seems that vevo video don't use encrypted signatures anymore.
Ismaël Mejía 38db46794f Merge branch 'ted_subtitles'
Ismaël Mejía 1f343eaabb [subtitles] refactor to support websites with subtitle information the
webpage.

I added the parameter webpage, so now it's similar to the way automatic
captions are handled. This is an improvement needed for websites like
TED.
Philipp Hagemeister 8eddf3e91d [youtube] Encode subtitle track name in request (Fixes )
Philipp Hagemeister 795f28f871 [youtube] Fix login (Fixes )
Jaime Marquínez Ferrándiz 600cc1a4f0 [youtube] Set the format_id field to the itag of the format (closes )
Jaime Marquínez Ferrándiz ea32fbacc8 Fix the extensions of two tests with youtube videos
The best quality is now a mp4 video.
Jaime Marquínez Ferrándiz 00fe14fc75 [youtube] Also use the 'adaptative_fmts' field from the /get_video_info page (fixes )
The 'adaptative_fmts' field from the video page is not added to the 'url_encoded_fmt_stream_map'
Philipp Hagemeister 6803655ced Merge pull request from rbrito/fix-extension
extractor: youtube: Set extension of AAC audio formats to m4a.
Jaime Marquínez Ferrándiz f6a54188c2 [youtube] Use 'node is None' when checking if the video has automatic captions
It had stopped working and it reports a FutureWarning
Rogério Brito d5a9bb4ea9 extractor: youtube: Swap video dimensions to match standard practice.
While working on this, I thought about simplifying things like changing
480x854 to 480p, and that seemed like a good option, until I realized that
people (me included) usually link the concept of some number followed by a p
with the video being 16:9.

So, we would be losing some information and, as we all know,
[explicit is better than implicit][*].

[*]: http://www.python.org/dev/peps/pep-0020/

This closes .

Signed-off-by: Rogério Brito <rbrito@ime.usp.br>
Rogério Brito f6f1fc9286 extractor: youtube: Fix extension of dash formats.
While we are at it, separate the audio formats from the video formats.

Signed-off-by: Rogério Brito <rbrito@ime.usp.br>
Rogério Brito 16f36a6fc9 extractor: youtube: Set extension of AAC audio formats to m4a.
This, in particular, eases downloading both audio and videos in DASH formats
before muxing them, which alleviates the problem that I exposed on issue

Furthermore, one may argue that this is, indeed, the case for correctness's
sake.

Signed-off-by: Rogério Brito <rbrito@ime.usp.br>
Philipp Hagemeister a733eb6c53 [youtube] Do not crash if caption info is missing altogether (Fixes )
Jai Grimshaw 1fb07d10a3 [youtube] Adds Download annotations
Adds  Download annotations from youtube
Annotations are downloaded and written to a .annotations.xml file using the https://www.youtube.com/annotations_invideo?features=1&legacy=1&video_id=$VIDEOID API.
Added unit test for annotations.
Philipp Hagemeister 57da92b7df [youtube] Do not recognize attribution link as user (Fixes )
Jaime Marquínez Ferrándiz a34c2faae4 [youtube] set the 'name' parameter in the subtitles url (fixes )
Philipp Hagemeister 15870e90b0 Restore warning when user forgets to quote URL ()
Philipp Hagemeister cfadd183c4 Call extracted property age_limit everywhere
Philipp Hagemeister 8dbe9899a9 Allow users to specify an age limit (fixes )
With these changes, users can now restrict what videos are downloaded by the intented audience, by specifying their age with --age-limit YEARS .
Add rudimentary support in youtube, pornotube, and youporn.
Philipp Hagemeister 7f8ae73a5d Include length in player cache ID
Some videos use the same player with IDs of multiple lengths.
See https://travis-ci.org/rg3/youtube-dl/jobs/12126506#L319 for an example.
Philipp Hagemeister c21315f273 [youtube] new static 82 signature
Philipp Hagemeister c38b1e776d [youtube] Simplify cache_dir code ()
Philipp Hagemeister 4f8bf17f23 Merge remote-tracking branch 'holomorph/master'
Philipp Hagemeister ca40186c75 [youtube] Fix static 82 signature (Closes )
Philipp Hagemeister a8c6b24155 [youtube] Support videos without a title (Fixes , Closes )
Filippo Valsorda bd8e5c7ca2 Merge pull request from rg3/no-playlist
[youtube] implement --no-playlist to only download current video
Filippo Valsorda 7c61bd36bb [youtube] correct --no-playlist for python3
Philipp Hagemeister 44d466559e Properly handle stream meap not being present
Philipp Hagemeister f10503db67 Handle videos without url_encoded_fmt_stream_map (Fixes )
Mark Oteiza fa55675593 Support XDG base directory specification
Filippo Valsorda 47192f92d8 implement --no-playlist to only download current video - closes
Jaime Marquínez Ferrándiz 9abb32045a [youtube] Add hlsvp to the error message if it can't be found and remove the live stream test
It's no longer available, other olympics streams have the same problem.
Jaime Marquínez Ferrándiz f490e77e77 [youtube] Set the thumbnail to None if it can't be extracted
Jaime Marquínez Ferrándiz 2dc592991a [youtube] update description of test
Philipp Hagemeister 920de7a27d [youtube] Fix 83 signature (Closes )
Philipp Hagemeister 695dc094ab Merge branch 'automatic-signatures'
Philipp Hagemeister f2c327fd39 Fix 86 signature ()
Philipp Hagemeister e35e4ddc9a Fix output of --youtube-print-sig-code when counting down to 0
Philipp Hagemeister c3c88a2664 Allow opts.cachedir == None to disable cache
Jaime Marquínez Ferrándiz bb0eee71e7 [youtube] Update one of the test's description
Jaime Marquínez Ferrándiz 6f56389b88 [youtube] update algos for length 86 and 84 (fixes )
Philipp Hagemeister c705320f48 Correct test strings
Philipp Hagemeister d2d8f89531 Do not warn if fallback is without alternatives (because we did not get the flash player URL)
Philipp Hagemeister bdde940e90 [youtube] Improve flash player URL handling
Philipp Hagemeister 13dc64ce74 [youtube] Remove _decrypt_signature_age_gate
Philipp Hagemeister f8061589e6 [youtube] Actually pass in cachedir option
Philipp Hagemeister 0ca96d48c7 [youtube] Improve source code quality
Philipp Hagemeister 4ba146f35d Update static signatures
Philipp Hagemeister edf3e38ebd [youtube] Improve cache and add an option to print the extracted signatures
Philipp Hagemeister c4417ddb61 [youtube] Add filesystem signature cache
Philipp Hagemeister 2f2ffea9ca Clarify a couple of calls
Philipp Hagemeister ba552f542f Use reader instead of indexing
Philipp Hagemeister 8379969834 Prepare signature function caching
Philipp Hagemeister 95dbd2f990 Change test target (Verified with node.js)
Philipp Hagemeister a7177865b1 Implement more opcodes
Philipp Hagemeister e0df6211cc Restore accidentally deleted commits
That's what happens if you let Windows machines write :(
Jaime Marquínez Ferrándiz 38d025b3f0 [youtube] add algo for length 91
patrickslin bc4b900898 Unable to decrypt signature length 93 (fixes )
Jaime Marquínez Ferrándiz 71c82637e7 [youtube] apply the fix for lists with number of videos multiple of _MAX_RESULTS to user extraction
Copied from the playlist extractor.
Jaime Marquínez Ferrándiz 4a67aafb7e [youtube] Don't search the flash player version for videos with age gate activated
Philipp Hagemeister 6ae8ee3f54 Update 85 signature (Fixes )
This is the first signature algorithm to have been parsed automatically, although that only works for HTML5 players for now, and is not yet integrated into master.
Jaime Marquínez Ferrándiz e69ae5b9e7 [youtube] support youtube.googleapis.com/v/* urls (fixes )
Jaime Marquínez Ferrándiz ce85f022d2 [youtube] update algo for length 82 (fixes )
Jaime Marquínez Ferrándiz e3dc22ca3a [youtube] Fix detection of videos with automatic captions
Jaime Marquínez Ferrándiz d665f8d3cb [subtitles] Also list the available automatic captions languages with '--list-sub'
Jaime Marquínez Ferrándiz 055e6f3657 [youtube] Support automatic captions with original language different from English (fixes ) and download in multiple languages.
Jaime Marquínez Ferrándiz 7fad1c6328 [subtitles] Use self._download_webpage for extracting the subtitles
It raises ExtractorError for the same exceptions we have to catch.
Jaime Marquínez Ferrándiz d82134c339 [subtitles] Simplify the extraction of subtitles in subclasses and remove NoAutoSubtitlesInfoExtractor
Subclasses just need to call the method extract_subtitles, which will call _extract_subtitles and _request_automatic_caption
Now the default implementation of _request_automatic_caption returns {}.
Jaime Marquínez Ferrándiz 54d39d8b2f [subtitles] rename SubitlesIE to SubtitlesInfoExtractor
Otherwise it can be automatically detected as a IE ready for use.
Jaime Marquínez Ferrándiz de7f3446e0 [youtube] move subtitles methods from the base extractor to YoutubeIE
Ismael Mejia f8e52269c1 [subtitles] made inheritance hierarchy flat as requested
Ismael Mejia cf1dd0c59e Merge branch 'master' into subtitles_rework
Jaime Marquínez Ferrándiz aa8f2641da [youtube] update algo for length 85 (fixes and fixes )
Jaime Marquínez Ferrándiz 8963d9c266 [youtube] Modify the regex to match ids of length 11 (fixes )
In urls like http://www.youtube.com/watch?v=BaW_jenozKcsharePLED17F32AD9753930 you can't split the query string and ids always have that length.
Jaime Marquínez Ferrándiz 890f62e868 Revert "[youtube] Fix detection of tags from HLS videos."
They have undo the change

This reverts commit 0638ad9999.
Ismael Mejia 72836fcee4 Merge branch 'master' into subtitles_rework
Ismael Mejia d6e203b3dc [subtitles] fixed multiple subtitles language separated by comma after merge
As mentioned in the pull request, I forgot to include this changes.
aa6a10c44a
Jaime Marquínez Ferrándiz e3ea479087 [youtube] Fix some issues with the detection of playlist/channel urls (reported in )
They were being caught by YoutubeUserIE, now it only extracts a url if the rest of extractors aren't suitable.
Now the url tests check that the urls can only be extracted with an specific extractor.
Jaime Marquínez Ferrándiz faab1d3836 [youtube] Fix detection of feeds urls (fixes )
Urls like https://www.youtube.com/feed/watch_later were being as users (before the last changes to YoutubeUserIE, as videos)