Commit Graph

369 Commits (964afd0689bdd7140b8ab182273d6379fe7b0548)

Author SHA1 Message Date
Philipp Hagemeister 62cd676c74 [youtube] Fixup DASH m4a headers
This fixes , , , , , .
Philipp Hagemeister 5f0d813d93 Merge remote-tracking branch 'rupertbaxter2/master'
Conflicts:
	youtube_dl/__init__.py
	youtube_dl/downloader/common.py
Philipp Hagemeister 083c9df93b [YoutubeDL] Allow filtering by properties (Fixes )
Pierre 4340decad2 check for overwriting files in the downloader (fixes , closes )
Jaime Marquínez Ferrándiz f3ff1a3696 YoutubeDL: Make the decision about removing the original file after each postprocessor is run (fixes )
If one of the processors said the file should be kept, it wouldn't pay
attention to the response from the following processors. This was wrong if the
'keep_video' option was False, if the first extractor modifies the original file
and then we extract its audio we don't want to keep the original video file.
Philipp Hagemeister 8bfa75451b [options] Add --no-call-home
While we're at it, also drop "experimental" moniker for --call-home - should work fine.
Philipp Hagemeister 58b1f00d19 [YoutubeDL] Add new --call-home option for debugging
Philipp Hagemeister be4a824d74 Add new option --source-address
Closes , fixes , fixes , fixes , closes .
Philipp Hagemeister 6271f1cad9 [youtube|ffmpeg] Automatically correct video with non-square pixels (Fixes )
Philipp Hagemeister 45598f1578 Merge remote-tracking branch 'aft90/merge-output-format'
Conflicts:
	youtube_dl/YoutubeDL.py
Andrei Troie d02115f837 Use the option in preparing the merge output filename
Philipp Hagemeister f90ad27375 [YoutubeDL] Copy over format metadata when merging (Fixes )
Philipp Hagemeister 0217c78377 [YoutubeDL] Allow selection by more extensions
Philipp Hagemeister 0590062925 Respect age_limit when listing extractors (Fixes )
Philipp Hagemeister e65566a9cc [youtube] Correct handling when DASH manifest is not necessary to find all formats
Jaime Marquínez Ferrándiz f00fd51dae Don't write the description file if info_dict['description'] is None ()
Jaime Marquínez Ferrándiz 0e4cb4f406 YoutubeDL: style fix
AndroKev cd58dc3e56 Update YoutubeDL.py
AndroKev c33bcf2051 only add video-id to archive, when successful
Example:
no space left--> youtube-dl adds the id to archive, but the video isn't fully downloaded
Philipp Hagemeister 734ea11e3c Drop hash character in downloader output ()
Philipp Hagemeister d86007873e [YoutubeDL] Document where details for format can be found
Philipp Hagemeister 71b640cc5b [YoutubeDL] Add declarative version of progress hooks
Philipp Hagemeister 4f026fafbc [YoutubeDL] Make postprocessors declarative
Instead of having to configure PPs in code, this allows us and embedding programs not to worry about imports or finer details, similarly to how we handle IEs.
Mark Schreiber ff815fe65a Download playlist items in reverse order
Series of videos are typically uploaded to YouTube playlists in
chronological order.  By default, these videos are downloaded
latest-to-earliest; this is great for seeing the latest videos in a
series, but prevents streaming video in the order that the videos were
produced.  Add an option to download videos in reverse order,
earliest-to-latest.

Conflicts:
	youtube_dl/YoutubeDL.py
	youtube_dl/__init__.py
Philipp Hagemeister 412c617d0f [cnet] Update to new theplatform infrastructure (Fixes )
Philipp Hagemeister f17e4c9c28 [screenwavemedia] Simplify ()
cryptonaut 16ae61f655 Handle --get-url with merged formats (fixes )
Outputs one URL per line
Philipp Hagemeister b82f815f37 Allow iterators for playlist result entries
Sergey M․ 706d7d4ee7 [YoutubeDL] Avoid negative timestamps on Windows
Philipp Hagemeister 4aae94f9d0 [YoutubeDL] Remove incorrect documentation
Philipp Hagemeister acda92f6bc Clarify --no-playlist documentation (Closes )
Jouke Waleson 9e1a5b8455 PEP8: applied even more rules
Jouke Waleson 8bcc875676 PEP8: more applied
Jouke Waleson 5f6a1245ff PEP8 applied
Philipp Hagemeister 7d4111ed14 Provide guidance when called with a YouTube ID starting with a dash.
Reported at https://news.ycombinator.com/item?id=8648121
Philipp Hagemeister 42e12102a9 [YoutubeDL] Fix multi_video check
Philipp Hagemeister dfd5313afd [YoutubeDL] Support new _type multi_video
Philipp Hagemeister c9bf41145f [YoutubeDL] Warn if an extractor returns compat_list
Jaime Marquínez Ferrándiz 92120217eb [cache] Fix writing to paths with unicode characters
* Use "compat_getenv"
* "write_json_file" now expects the filename to be a string
Philipp Hagemeister c6afed48ff [YoutubeDL] guard against strange sys.stdouts
Jaime Marquínez Ferrándiz c295490830 [YoutubeDL] Fix bug in the detection of formats that don't contain video (fixes )
If the format requested was not available, we called the method '.get' in None.
Philipp Hagemeister a1cf99d03a [YoutubeDL] Add playlist_id and playlist_title fields (Fixes )
Jaime Marquínez Ferrándiz f866e474f3 [YoutubeDL] Don't dowload formats for merging if the first doesn't contain the video ()
Philipp Hagemeister 8c25f81bee [util] Move compatibility functions out of util
utils is large enough without these compatibility functions.

Everything that is present in newer versions of Python (i.e. with dev Python it's just an import) goes into compat.py .
Everything else (i.e. youtube-dl-specific helpers) goes into utils.py .
Philipp Hagemeister 4c83c96795 [YoutubeDL] Include rtmpdump in exe versions -v output
Philipp Hagemeister fbb21cf528 [youtube] Add formats 298, 299 (Fixes )
Philipp Hagemeister 3511266bc3 [YoutubeDL] Simplify API of YoutubeDL
Calling add_default_extractors twice should be harmless since the first set of extractors will match.
Philipp Hagemeister 488447455d [ffmpeg] Warn if ffmpeg/avconv version is too old (Fixes )
Philipp Hagemeister d28b517154 [YoutubeDL] Output avconv/ffmpeg versions if -v is given
Sergey M․ f889cea109 Merge branch 'compat-getenv-and-expanduser' of https://github.com/dstftw/youtube-dl into dstftw-compat-getenv-and-expanduser
Conflicts:
	test/test_utils.py
	youtube_dl/__init__.py
Philipp Hagemeister 63e0be3415 New option --dump-single-json ()
Philipp Hagemeister e82c1e9a6e [YoutubeDL] Do not apply playlist info to videos when extract_flat is set ()
Philipp Hagemeister 057a5206cc Add --flat-playlist option (Closes )
George Boyle 53d9009bdb KeyError on initialising YoutubeDL in python3
George Boyle 1b725173a5 Fixed typo
Sergey M․ 4644ac5527 [core] Decode environment variables with filesystem encoding (Fixes , Fixes , Fixes )
Introduces compat versions of os.getenv and os.path.expanduser
Sergey M․ ee0d90707a [YoutubeDL] Fix string check for python3
Sergey M․ 68b0973046 [YoutubeDL] Expect all kind of strings in urlopen
Now it doesn't fail if req is python2's str
Philipp Hagemeister 1de33fafd9 [YoutubeDL] Allow downloading multiple formats with ,
Philipp Hagemeister e2e5dae64d Add -f m4a
Sergey M․ d05cfe0600 [YoutubeDL/utils] Clarify rationale for URL escaping in comment, move escape routines to utils and add some tests
Sergey M․ 37419b4f99 [YoutubeDL] Escape non-ASCII characters in URLs
urllib chokes on URLs with non-ASCII characters (see http://bugs.python.org/issue3991)
Working around by replacing request's original URL with escaped one
Philipp Hagemeister a0e07d3161 [youtube] Move cache into its own module
Philipp Hagemeister 241f7a8ade Merge remote-tracking branch 'JGjorgji/fix-leading-zeroes'
Philipp Hagemeister 8d31fa3cce [execafterdownload] Simplify ()
Gjorgji Jankovski c6b4132a0a renamed for consistency
Gjorgji Jankovski ad260c90ab Filenames are padded according to the playlist length
Philipp Hagemeister be843678b1 [YouTubeDL] Correct handling of age_limit = None in result
Philipp Hagemeister e8ee972c6e Allow playlist test definitions in test_download.
This moves playlist tests where they belong, i.e. to the extractors themselves.
Additionally, all our network interaction configuration for tests in test_download now applies to playlist tests as well.
Philipp Hagemeister 11b85ce62e [YouTubeDL] Best practices (Closes )
Philipp Hagemeister 4192b51c7c Replace failure handling with up-front check.
The only time that write_string should fail is if the Python is completely braindead.
Check for that condition and output a more accurate warning.

See  for details.
rupertbaxter2 b7f8116406 Deletes temp files after postprocess merge unless -k option is specified
Philipp Hagemeister b0472057a3 [YoutubeDL] Make sure we really, really get out the encoding string
Fixes 
Apparently, on some platforms, even outputting this fails already.
Philipp Hagemeister 9732d77ed2 [snotr] PEP8 and minor fixes ()
Philipp Hagemeister 8d5797b00f [YoutubeDL] Show download URL when -v is set
This will allow us to debug issues like 
Philipp Hagemeister be6d722904 [cnn] Improve thumbnail extraction
Philipp Hagemeister d551980823 [spiegeltv] Simplify and PEP8
Philipp Hagemeister acd69589a5 [YoutubeDL] Do not require default output template to be set
Philipp Hagemeister c57f775710 [YoutubeDL] Add simple tests for format_note (Closes )
robbie e4db19511a Fix subtitle download error reporting (Fixes )
Philipp Hagemeister b58ddb32ba [utils] Completely rewrite Windows output (Fixes )
Philipp Hagemeister 734f90bb41 Use --encoding when outputting
Philipp Hagemeister d26e981df4 Correct check for empty dirname (Fixes )
Philipp Hagemeister bec1fad223 [YouTubeDL] Throw an early error if the info_dict result is invalid
Philipp Hagemeister cce929eaac [franceculture] Add extractor (Fixes )
Philipp Hagemeister 62fec3b2ff Add new --encoding option (Fixes )
Philipp Hagemeister ad8915b729 Add --no-warnings option (Fixes )
Philipp Hagemeister ea38e55fff [instagram] Add support for user profiles (Fixes )
Philipp Hagemeister 7e8c0af004 Add --prefer-insecure option (Fixes )
Philipp Hagemeister 1a4895453a [YoutubeDL] Improve error message
Jaime Marquínez Ferrándiz bc6d597828 Add bestvideo and worstvideo to special format names ()
Philipp Hagemeister 955c451456 Rename upload_timestamp to timestamp
Philipp Hagemeister 9d2ecdbc71 [vevo] Centralize timestamp handling
Philipp Hagemeister db95dc13a1 [playvid] Simplify ()
Jaime Marquínez Ferrándiz a0792b738e Don't install the global url opener
All the code uses now the urlopen method of YoutubeDL
Jaime Marquínez Ferrándiz 19a41fc613 Don't set the global socket timeout
Use the timeout argument of the `OpenerDirector.open` method instead
Philipp Hagemeister 805ef3c60b Correct automatic resolution determination
Jaime Marquínez Ferrándiz 6d07ce0162 YoutubeDL: If the logger is set call its `warning` method in `report_warning`
Jaime Marquínez Ferrándiz e9c092f125 YoutubeDL: Use its `urlopen` method for downloading the thumbnail.
Philipp Hagemeister 17b75c0de1 Document width, height, and resolution ()
Philipp Hagemeister c9ae7b9565 [youtube] Add support for search result URLs (Fixes )
Philipp Hagemeister 0afef30b23 Add display_id field
Jaime Marquínez Ferrándiz f89197d73e Some pep8 style fixes
Philipp Hagemeister 1394ce65b4 [youtube] Add new formats (Fixes )
Philipp Hagemeister fd28827864 Do not count unmatched videos for --max-downloads (Fixes )
Philipp Hagemeister d3e5bbf437 Correct --max-downloads with --ignore-errors
Philipp Hagemeister 65697b3bf3 Merge branch 'paged-lists'
Conflicts:
	test/test_utils.py
	youtube_dl/extractor/youtube.py
Philipp Hagemeister 50317b111d Merge branch 'youtube-dash-manifest'
Conflicts:
	youtube_dl/extractor/youtube.py
Philipp Hagemeister de3ef3ed58 Default to -f best-audio when only audio is requested
Philipp Hagemeister ba7678f9cc Add -f bestaudio (Fixes )
Philipp Hagemeister 04b4d394d9 Add new --default-search option ()
Philipp Hagemeister 7b0817e8e1 [servingsys] Add support
This also adds support for brightcove advertisements.
Fixes 
Philipp Hagemeister b7ab059084 Add infrastructure for paged lists
This commit allows to download pages in playlists as needed instead of all at once.
Before this commit,
    youtube-dl http://www.youtube.com/user/ANNnewsCH/videos --playlist-end 2 --skip-download
took quite some time - now it's almost instantaneous.
As an example, the youtube:user extractor has been converted.
Fixes 
Philipp Hagemeister dd27fd1739 [youtube] Download DASH manifest
If given, download and parse the DASH manifest file, in order to get ultra-HQ formats.
Fixes 
Jaime Marquínez Ferrándiz 58c3c7ae38 Don’t try to merge the formats if ffmpeg or avconv are not installed
Jaime Marquínez Ferrándiz 76b1bd672d Add ‘--prefer-avconv’ and ‘--prefer-ffmpeg’ options ()
Affects the ffmpeg post processors, if ‘--prefer-ffmpeg’ is given and both avconv and ffmpeg are installed, it will use ffmpeg. Otherwise it will follow the old behaviour.
Philipp Hagemeister 282962bd36 --list-formats: Only add "@" if vbr is given
Philipp Hagemeister 56327689a2 Move postprocessor into its own package
Philipp Hagemeister a9c5e5ca6e Set required properties for format merging
Philipp Hagemeister a015dce0e2 Merge remote-tracking branch 'jaimeMF/merge-formats'
Philipp Hagemeister 6febd1c1df Prepare widespread unicode literal use
Jaime Marquínez Ferrándiz 6350728be2 Allow merging formats (closes )
Multiple formats can be requested using `-f 137+139`, each one is downloaded and then the two are merged with ffmpeg.
Philipp Hagemeister eadaf08c16 Merge remote-tracking branch 'origin/master'
Jaime Marquínez Ferrándiz 241bce7aaf Merge pull request from rzhxeo/var
Correct variable name in YoutubeDL.list_formats
Philipp Hagemeister 33ec2ae8d9 Merge remote-tracking branch 'origin/master'
Jaime Marquínez Ferrándiz e9f9a10fba Fix initialization of YoutubeDL with params set to None
Set it to an empty dictionary because it’s directly accessed when setting some properties
rzhxeo 1cdfc31e1f Correct variable name in YoutubeDL
Philipp Hagemeister a0ddb8a2fa Add new --print-traffic option
Philipp Hagemeister 7217e148fb [yahoo] Use centralized sorting, and add tbr field
Philipp Hagemeister c7deaa4c74 [zdf] Use centralized sorting
Philipp Hagemeister 4bcc7bd1f2 Add temporary _sort_formats helper function
Philipp Hagemeister f49d89ee04 Add a resolution field and improve general --list-formats output
Philipp Hagemeister 1538eff6d8 [bliptv] Remove support for direct downloads
This is now handled by the generic IE
Jaime Marquínez Ferrándiz 933605d7e8 YoutubeDL: rename `_fd_progress_hooks` back to `_progress_hooks`
In the future it may report more things.
Jaime Marquínez Ferrándiz b3d9ef88ec YoutubeDL: only set the ‘formats’ field of the info_dict if it was already set before
It caused a circular reference error, when trying to dump it to json (for example with the test video for myvideo.de or any other video without formats)
Philipp Hagemeister 9fc3bef87a Merge remote-tracking branch 'jaimeMF/split-downloaders'
Philipp Hagemeister f82b18efc1 Merge remote-tracking branch 'rzhxeo/youtube'
Philipp Hagemeister 196938835a Remove debugging code
Introduced by accident in 5d681e960d
Philipp Hagemeister 5d681e960d Use bidiv instead of fribidi if available (Fixes )
rzhxeo 62d68c43ed Make prefer_free_formats sorting more robust
rzhxeo bfaae0a768 Filter and sort videos before calling list_formats
rzhxeo 4ea3be0a5c [YoutubeIE] Externalize format selection
Philipp Hagemeister 29eb517403 Add webpage_url_basename info_dict field (Fixes )
alimirjamali 0a9ce268ba Incorrect variable is used to check whether thumbnail exists
Dear @phihag

I believe in line 848, the correct variable to check is 'thumb_filename' rather than 'infofn'

Kindly advise

Mit freundlichen Gruessen
Ali
Philipp Hagemeister a19fd00cc4 Simplify --playlist-start / --playlist-end interface
Philipp Hagemeister 7b6fefc9d4 Apply --no-overwrites for --write-* files as well (Fixes )
Philipp Hagemeister 525ef9227f Add --get-duration (Fixes )
Philipp Hagemeister 5fe18bdbde Add --min-views / --max-views (Fixes )
Jaime Marquínez Ferrándiz 3bc2ddccc8 Move FileDownloader to its own module and create a new class for each download process
A suitable downloader can be found using the 'get_suitable_downloader' function.

Each subclass implements 'real_download', for downloading an info dict you call the 'download' method, which first checks if the video has already been downloaded
Jaime Marquínez Ferrándiz 8ab470f1b2 Now a new FileDownloader is created when downloading a video
The progress hooks can be added using the method "add_downloader_progress_hook"