c84dd8a90d
[YoutubeDL] store the subtitles to download in the 'requested_subtitles' field
...
We need to keep the orginal subtitles information, so that the '--load-info' option can be used to list or select the subtitles again.
We'll also be able to have a separate field for storing the automatic captions info.
10 years ago
a504ced097
Improve subtitles support
...
For each language the extractor builds a list with the available formats sorted (like for video formats), then YoutubeDL selects one of them using the '--sub-format' option which now allows giving the format preferences (for example 'ass/srt/best').
For each format the 'url' field can be set so that we only download the contents if needed, or if the contents needs to be processed (like in crunchyroll) the 'data' field can be used.
The reasons for this change are:
* We weren't checking that the format given with '--sub-format' was available, checking it in each extractor would be repetitive.
* It allows to easily support giving a format preference.
* The subtitles were automatically downloaded in the extractor, but I think that if you use for example the '--dump-json' option you want to finish as fast as possible.
Currently only the ted extractor has been updated, but the old system still works.
10 years ago
73fac4e911
[ffmpeg] Add --ffmpeg-location
10 years ago
442c37b7a9
[YoutubeDL] Do not perform filter matching on partial results ( Fixes #4921 )
10 years ago
7e5db8c930
[options] Add --no-color
10 years ago
347de4931c
[YoutubeDL] Add generic video filtering ( Fixes #4916 )
...
This functionality is intended to eventually encompass the current format filtering.
10 years ago
43f244b6d5
[YoutubeDL] Do not show worst in --list-formats output
...
Nobody wants to know what the worst possible format is. And if they do, they can still provide -f worst.
10 years ago
2ec19e9558
[YoutubeDL] Allow filtering by audio sampling rate
10 years ago
9ddb6925bf
[YoutubeDL] Allow filtering by string properties ( #4906 )
10 years ago
b3a286d69d
[YoutubeDL] _calc_cookies: add get_header method to _PseudoRequest ( #4861 )
10 years ago
300753a069
[YoutubeDL] Fix video+audio format field ( Closes #4880 )
10 years ago
f6c24009be
[YoutubeDL] Calculate thumbnail IDs automatically
10 years ago
8f9312c387
Appease pyflakes8-3
10 years ago
6d593c3276
[YoutubeDL] Fix video+audio format_id ( Closes #4824 )
10 years ago
5468ff4d91
Remove duplicate dictionary keys
10 years ago
a7a14d9586
[YoutubeDL] set the 'thumbnails' field if the info_dict has the 'thumbnails' field
...
Since the '--write-thumbnail' uses the 'thumbnails' field and we didn't updated the info_dict, it wouldn't detect the thumbnail. (fixes #4812 )
10 years ago
dcf53d4408
[YoutubeDL] Set format_id for video+audio ( Closes #3634 )
10 years ago
3a0d2f520a
[YoutubeDL] Temporary fix for subprocess encoding issues on python2 @ Windows ( Closes #4787 )
...
For now filenames will be encoded with preferrefencoding before written to disk
10 years ago
9f0df77ab1
[YoutubeDL] Allow format filtering by fps
10 years ago
e72c7e4123
[YoutubeDL] Always set the '_filename' field in the info_dict (reported in #4053 )
...
It's also useful when you use the '--write-info-json' option.
10 years ago
ee69b99af6
[YoutubeDL] clarify hook documentation
10 years ago
881e6a1f5c
Add --xattr-set-filesize option ( Fixes #1348 )
10 years ago
c14e88f0f5
[YoutubeDL] Add --playlist-items option ( Fixes #2662 )
10 years ago
ec82d85acd
[YoutubeDL] Implement --write-all-thumbnails ( Closes #2269 )
10 years ago
cfb56d1af3
Add --list-thumbnails
10 years ago
1070711d60
[YoutubeDL._calc_cookies] Restore the 'is_unverifiable'
...
I should have check everything was copied before commiting 4b405cfc6e
10 years ago
4b405cfc6e
[YoutubeDL._calc_cookies] Restore the 'has_header' method
...
I didn't copied it from downloader/external
10 years ago
e5660ee6ae
[YoutubeDL] Fill the info dict 'http_headers' field with all the headers available
...
Useful for external tools using the json output.
The methods '_calc_headers' and '_calc_cookies' have been copied from the downloader/external, now they just use "info_dict['http_headers']".
10 years ago
222516d97d
[downloader] Lay groundwork for external downloaders.
...
This comes with a very simply implementation for wget; the real work is in setting up the infrastructure.
10 years ago
a055469faf
[downloader] Improve downloader selection
10 years ago
62cd676c74
[youtube] Fixup DASH m4a headers
...
This fixes #2288 , #2506 , #2607 , #3681 , #4741 , #4767 .
10 years ago
5f0d813d93
Merge remote-tracking branch 'rupertbaxter2/master'
...
Conflicts:
youtube_dl/__init__.py
youtube_dl/downloader/common.py
10 years ago
083c9df93b
[YoutubeDL] Allow filtering by properties ( Fixes #4584 )
10 years ago
4340decad2
check for overwriting files in the downloader ( fixes #3916 , closes #3829 )
10 years ago
f3ff1a3696
YoutubeDL: Make the decision about removing the original file after each postprocessor is run ( fixes #2261 )
...
If one of the processors said the file should be kept, it wouldn't pay
attention to the response from the following processors. This was wrong if the
'keep_video' option was False, if the first extractor modifies the original file
and then we extract its audio we don't want to keep the original video file.
10 years ago
8bfa75451b
[options] Add --no-call-home
...
While we're at it, also drop "experimental" moniker for --call-home - should work fine.
10 years ago
58b1f00d19
[YoutubeDL] Add new --call-home option for debugging
10 years ago
be4a824d74
Add new option --source-address
...
Closes #3618 , fixes #721 , fixes #2481 , fixes #4551 , closes #1020 .
10 years ago
6271f1cad9
[youtube|ffmpeg] Automatically correct video with non-square pixels ( Fixes #4674 )
10 years ago
45598f1578
Merge remote-tracking branch 'aft90/merge-output-format'
...
Conflicts:
youtube_dl/YoutubeDL.py
10 years ago
d02115f837
Use the option in preparing the merge output filename
10 years ago
f90ad27375
[YoutubeDL] Copy over format metadata when merging ( Fixes #4671 )
10 years ago
0217c78377
[YoutubeDL] Allow selection by more extensions
10 years ago
0590062925
Respect age_limit when listing extractors ( Fixes #4653 )
10 years ago
e65566a9cc
[youtube] Correct handling when DASH manifest is not necessary to find all formats
10 years ago
f00fd51dae
Don't write the description file if info_dict['description'] is None ( #3166 )
10 years ago
0e4cb4f406
YoutubeDL: style fix
10 years ago
cd58dc3e56
Update YoutubeDL.py
10 years ago
c33bcf2051
only add video-id to archive, when successful
...
Example:
no space left--> youtube-dl adds the id to archive, but the video isn't fully downloaded
10 years ago
734ea11e3c
Drop hash character in downloader output ( #4484 )
10 years ago
d86007873e
[YoutubeDL] Document where details for format can be found
10 years ago
71b640cc5b
[YoutubeDL] Add declarative version of progress hooks
10 years ago
4f026fafbc
[YoutubeDL] Make postprocessors declarative
...
Instead of having to configure PPs in code, this allows us and embedding programs not to worry about imports or finer details, similarly to how we handle IEs.
10 years ago
ff815fe65a
Download playlist items in reverse order
...
Series of videos are typically uploaded to YouTube playlists in
chronological order. By default, these videos are downloaded
latest-to-earliest; this is great for seeing the latest videos in a
series, but prevents streaming video in the order that the videos were
produced. Add an option to download videos in reverse order,
earliest-to-latest.
Conflicts:
youtube_dl/YoutubeDL.py
youtube_dl/__init__.py
10 years ago
412c617d0f
[cnet] Update to new theplatform infrastructure ( Fixes #2736 )
10 years ago
f17e4c9c28
[screenwavemedia] Simplify ( #3766 )
10 years ago
16ae61f655
Handle --get-url with merged formats ( fixes #2883 )
...
Outputs one URL per line
10 years ago
b82f815f37
Allow iterators for playlist result entries
10 years ago
706d7d4ee7
[YoutubeDL] Avoid negative timestamps on Windows
10 years ago
4aae94f9d0
[YoutubeDL] Remove incorrect documentation
10 years ago
acda92f6bc
Clarify --no-playlist documentation ( Closes #4309 )
10 years ago
9e1a5b8455
PEP8: applied even more rules
10 years ago
8bcc875676
PEP8: more applied
10 years ago
5f6a1245ff
PEP8 applied
10 years ago
7d4111ed14
Provide guidance when called with a YouTube ID starting with a dash.
...
Reported at https://news.ycombinator.com/item?id=8648121
10 years ago
42e12102a9
[YoutubeDL] Fix multi_video check
10 years ago
dfd5313afd
[YoutubeDL] Support new _type multi_video
10 years ago
c9bf41145f
[YoutubeDL] Warn if an extractor returns compat_list
10 years ago
92120217eb
[cache] Fix writing to paths with unicode characters
...
* Use "compat_getenv"
* "write_json_file" now expects the filename to be a string
10 years ago
c6afed48ff
[YoutubeDL] guard against strange sys.stdouts
10 years ago
c295490830
[YoutubeDL] Fix bug in the detection of formats that don't contain video ( fixes #4150 )
...
If the format requested was not available, we called the method '.get' in None.
10 years ago
a1cf99d03a
[YoutubeDL] Add playlist_id and playlist_title fields ( Fixes #4139 )
10 years ago
f866e474f3
[YoutubeDL] Don't dowload formats for merging if the first doesn't contain the video ( #4132 )
10 years ago
8c25f81bee
[util] Move compatibility functions out of util
...
utils is large enough without these compatibility functions.
Everything that is present in newer versions of Python (i.e. with dev Python it's just an import) goes into compat.py .
Everything else (i.e. youtube-dl-specific helpers) goes into utils.py .
10 years ago
4c83c96795
[YoutubeDL] Include rtmpdump in exe versions -v output
10 years ago
fbb21cf528
[youtube] Add formats 298, 299 ( Fixes #4056 )
10 years ago
3511266bc3
[YoutubeDL] Simplify API of YoutubeDL
...
Calling add_default_extractors twice should be harmless since the first set of extractors will match.
10 years ago
488447455d
[ffmpeg] Warn if ffmpeg/avconv version is too old ( Fixes #4026 )
10 years ago
d28b517154
[YoutubeDL] Output avconv/ffmpeg versions if -v is given
10 years ago
f889cea109
Merge branch 'compat-getenv-and-expanduser' of https://github.com/dstftw/youtube-dl into dstftw-compat-getenv-and-expanduser
...
Conflicts:
test/test_utils.py
youtube_dl/__init__.py
10 years ago
63e0be3415
New option --dump-single-json ( #4003 )
10 years ago
e82c1e9a6e
[YoutubeDL] Do not apply playlist info to videos when extract_flat is set ( #4003 )
10 years ago
057a5206cc
Add --flat-playlist option ( Closes #4003 )
10 years ago
53d9009bdb
KeyError on initialising YoutubeDL in python3 #3910
10 years ago
1b725173a5
Fixed typo
10 years ago
4644ac5527
[core] Decode environment variables with filesystem encoding ( Fixes #3854 , Fixes #3217 , Fixes #2918 )
...
Introduces compat versions of os.getenv and os.path.expanduser
10 years ago
ee0d90707a
[YoutubeDL] Fix string check for python3
10 years ago
68b0973046
[YoutubeDL] Expect all kind of strings in urlopen
...
Now it doesn't fail if req is python2's str
10 years ago
1de33fafd9
[YoutubeDL] Allow downloading multiple formats with ,
10 years ago
e2e5dae64d
Add -f m4a
10 years ago
d05cfe0600
[YoutubeDL/utils] Clarify rationale for URL escaping in comment, move escape routines to utils and add some tests
10 years ago
37419b4f99
[YoutubeDL] Escape non-ASCII characters in URLs
...
urllib chokes on URLs with non-ASCII characters (see http://bugs.python.org/issue3991 )
Working around by replacing request's original URL with escaped one
10 years ago
a0e07d3161
[youtube] Move cache into its own module
10 years ago
241f7a8ade
Merge remote-tracking branch 'JGjorgji/fix-leading-zeroes'
10 years ago
8d31fa3cce
[execafterdownload] Simplify ( #3569 )
10 years ago
c6b4132a0a
renamed for consistency
10 years ago
ad260c90ab
Filenames are padded according to the playlist length
10 years ago
be843678b1
[YouTubeDL] Correct handling of age_limit = None in result
10 years ago
e8ee972c6e
Allow playlist test definitions in test_download.
...
This moves playlist tests where they belong, i.e. to the extractors themselves.
Additionally, all our network interaction configuration for tests in test_download now applies to playlist tests as well.
11 years ago
11b85ce62e
[YouTubeDL] Best practices ( Closes #3370 )
11 years ago
4192b51c7c
Replace failure handling with up-front check.
...
The only time that write_string should fail is if the Python is completely braindead.
Check for that condition and output a more accurate warning.
See #3326 for details.
11 years ago
b7f8116406
Deletes temp files after postprocess merge unless -k option is specified
11 years ago
b0472057a3
[YoutubeDL] Make sure we really, really get out the encoding string
...
Fixes #3326
Apparently, on some platforms, even outputting this fails already.
11 years ago
9732d77ed2
[snotr] PEP8 and minor fixes ( #3296 )
11 years ago
8d5797b00f
[YoutubeDL] Show download URL when -v is set
...
This will allow us to debug issues like #3204
11 years ago
be6d722904
[cnn] Improve thumbnail extraction
11 years ago
d551980823
[spiegeltv] Simplify and PEP8
11 years ago
acd69589a5
[YoutubeDL] Do not require default output template to be set
11 years ago
c57f775710
[YoutubeDL] Add simple tests for format_note ( Closes #2825 )
11 years ago
e4db19511a
Fix subtitle download error reporting ( Fixes #2724 )
11 years ago
b58ddb32ba
[utils] Completely rewrite Windows output ( Fixes #2672 )
11 years ago
734f90bb41
Use --encoding when outputting
11 years ago
d26e981df4
Correct check for empty dirname ( Fixes #2683 )
11 years ago
bec1fad223
[YouTubeDL] Throw an early error if the info_dict result is invalid
11 years ago
cce929eaac
[franceculture] Add extractor ( Fixes #2669 )
11 years ago
62fec3b2ff
Add new --encoding option ( Fixes #2650 )
11 years ago
ad8915b729
Add --no-warnings option ( Fixes #2630 )
11 years ago
ea38e55fff
[instagram] Add support for user profiles ( Fixes #2606 )
11 years ago
7e8c0af004
Add --prefer-insecure option ( Fixes #2364 )
11 years ago
1a4895453a
[YoutubeDL] Improve error message
11 years ago
bc6d597828
Add bestvideo and worstvideo to special format names ( #2163 )
11 years ago
955c451456
Rename upload_timestamp to timestamp
11 years ago
9d2ecdbc71
[vevo] Centralize timestamp handling
11 years ago
db95dc13a1
[playvid] Simplify ( #2539 )
11 years ago
a0792b738e
Don't install the global url opener
...
All the code uses now the urlopen method of YoutubeDL
11 years ago
19a41fc613
Don't set the global socket timeout
...
Use the timeout argument of the `OpenerDirector.open` method instead
11 years ago
805ef3c60b
Correct automatic resolution determination
11 years ago
6d07ce0162
YoutubeDL: If the logger is set call its `warning` method in `report_warning`
11 years ago
e9c092f125
YoutubeDL: Use its `urlopen` method for downloading the thumbnail.
11 years ago
17b75c0de1
Document width, height, and resolution ( #1445 )
11 years ago
c9ae7b9565
[youtube] Add support for search result URLs ( Fixes #2495 )
11 years ago
0afef30b23
Add display_id field
11 years ago
f89197d73e
Some pep8 style fixes
11 years ago
1394ce65b4
[youtube] Add new formats ( Fixes #2221 )
11 years ago
fd28827864
Do not count unmatched videos for --max-downloads ( Fixes #2211 )
11 years ago
d3e5bbf437
Correct --max-downloads with --ignore-errors
11 years ago
65697b3bf3
Merge branch 'paged-lists'
...
Conflicts:
test/test_utils.py
youtube_dl/extractor/youtube.py
11 years ago
50317b111d
Merge branch 'youtube-dash-manifest'
...
Conflicts:
youtube_dl/extractor/youtube.py
11 years ago
de3ef3ed58
Default to -f best-audio when only audio is requested
11 years ago
ba7678f9cc
Add -f bestaudio ( Fixes #2163 )
11 years ago
04b4d394d9
Add new --default-search option ( #2193 )
11 years ago
7b0817e8e1
[servingsys] Add support
...
This also adds support for brightcove advertisements.
Fixes #2181
11 years ago
b7ab059084
Add infrastructure for paged lists
...
This commit allows to download pages in playlists as needed instead of all at once.
Before this commit,
youtube-dl http://www.youtube.com/user/ANNnewsCH/videos --playlist-end 2 --skip-download
took quite some time - now it's almost instantaneous.
As an example, the youtube:user extractor has been converted.
Fixes #2175
11 years ago
dd27fd1739
[youtube] Download DASH manifest
...
If given, download and parse the DASH manifest file, in order to get ultra-HQ formats.
Fixes #2166
11 years ago
58c3c7ae38
Don’t try to merge the formats if ffmpeg or avconv are not installed
11 years ago
76b1bd672d
Add ‘--prefer-avconv’ and ‘--prefer-ffmpeg’ options ( #2115 )
...
Affects the ffmpeg post processors, if ‘--prefer-ffmpeg’ is given and both avconv and ffmpeg are installed, it will use ffmpeg. Otherwise it will follow the old behaviour.
11 years ago
282962bd36
--list-formats: Only add "@" if vbr is given
11 years ago
56327689a2
Move postprocessor into its own package
11 years ago
a9c5e5ca6e
Set required properties for format merging
11 years ago
a015dce0e2
Merge remote-tracking branch 'jaimeMF/merge-formats'
11 years ago
6febd1c1df
Prepare widespread unicode literal use
11 years ago
6350728be2
Allow merging formats ( closes #1612 )
...
Multiple formats can be requested using `-f 137+139`, each one is downloaded and then the two are merged with ffmpeg.
11 years ago
eadaf08c16
Merge remote-tracking branch 'origin/master'
11 years ago
241bce7aaf
Merge pull request #2061 from rzhxeo/var
...
Correct variable name in YoutubeDL.list_formats
11 years ago
33ec2ae8d9
Merge remote-tracking branch 'origin/master'
11 years ago
e9f9a10fba
Fix initialization of YoutubeDL with params set to None
...
Set it to an empty dictionary because it’s directly accessed when setting some properties
11 years ago
1cdfc31e1f
Correct variable name in YoutubeDL
11 years ago
a0ddb8a2fa
Add new --print-traffic option
11 years ago
7217e148fb
[yahoo] Use centralized sorting, and add tbr field
11 years ago
c7deaa4c74
[zdf] Use centralized sorting
11 years ago
4bcc7bd1f2
Add temporary _sort_formats helper function
11 years ago
f49d89ee04
Add a resolution field and improve general --list-formats output
11 years ago
1538eff6d8
[bliptv] Remove support for direct downloads
...
This is now handled by the generic IE
11 years ago
933605d7e8
YoutubeDL: rename `_fd_progress_hooks` back to `_progress_hooks`
...
In the future it may report more things.
11 years ago
b3d9ef88ec
YoutubeDL: only set the ‘formats’ field of the info_dict if it was already set before
...
It caused a circular reference error, when trying to dump it to json (for example with the test video for myvideo.de or any other video without formats)
11 years ago
9fc3bef87a
Merge remote-tracking branch 'jaimeMF/split-downloaders'
11 years ago
f82b18efc1
Merge remote-tracking branch 'rzhxeo/youtube'
11 years ago
196938835a
Remove debugging code
...
Introduced by accident in 5d681e960d
11 years ago
5d681e960d
Use bidiv instead of fribidi if available ( Fixes #1912 )
11 years ago
62d68c43ed
Make prefer_free_formats sorting more robust
11 years ago
bfaae0a768
Filter and sort videos before calling list_formats
11 years ago
4ea3be0a5c
[YoutubeIE] Externalize format selection
11 years ago
29eb517403
Add webpage_url_basename info_dict field ( Fixes #1938 )
11 years ago
0a9ce268ba
Incorrect variable is used to check whether thumbnail exists
...
Dear @phihag
I believe in line 848, the correct variable to check is 'thumb_filename' rather than 'infofn'
Kindly advise
Mit freundlichen Gruessen
Ali
11 years ago
a19fd00cc4
Simplify --playlist-start / --playlist-end interface
11 years ago
7b6fefc9d4
Apply --no-overwrites for --write-* files as well ( Fixes #1980 )
11 years ago
525ef9227f
Add --get-duration ( Fixes #859 )
11 years ago
5fe18bdbde
Add --min-views / --max-views ( Fixes #1979 )
11 years ago
3bc2ddccc8
Move FileDownloader to its own module and create a new class for each download process
...
A suitable downloader can be found using the 'get_suitable_downloader' function.
Each subclass implements 'real_download', for downloading an info dict you call the 'download' method, which first checks if the video has already been downloaded
11 years ago
8ab470f1b2
Now a new FileDownloader is created when downloading a video
...
The progress hooks can be added using the method "add_downloader_progress_hook"
11 years ago
357ddadbf5
Fix thumbnail filename determination ( Fixes #1945 )
11 years ago
45598aab08
[YoutubeDL] Simplify filename preparation
11 years ago
26e6393134
Set 'NA' as the default value for missing fields in the output template ( fixes #1931 )
...
Remove the `except KeyError` clause, it won't get raised anymore
11 years ago
1c088fa89d
Improve --bidi-workaround support
11 years ago
395293a889
[--load-info] Always read file as UTF-8
...
This allows editing the file (and not escaping non-ASCII characters) and reloading it in.
11 years ago
db4da14027
Merge remote-tracking branch 'jaimeMF/load-info'
11 years ago
2101830c0d
Remove unused imports
11 years ago
a0d96c9843
Add filename to --dump-json output ( Fixes #1908 )
11 years ago
0783b09b92
Add a workaround for terminals without bidi support ( Fixes #1912 )
11 years ago
ef4fd84857
[wistia] Add extractor
11 years ago
7fc3fa0545
[9gag] Add extractor
11 years ago
d494389821
Option '--load-info': if the download fails, try extracting the info with the 'webpage_url' field of the info dict
...
The video url may have expired.
11 years ago
1dcc4c0cad
Add --load-info option ( #972 )
...
It just calls the 'YoutubeDL.process_ie_result' with the dictionary from the json file
11 years ago
6ad14cab59
Add --socket-timeout option
11 years ago
e344693b65
Make socket timeout configurable, and bump default to 10 minutes ( #1862 )
11 years ago
a3fb4675fb
Do not mutate default arguments
...
In this case, it looks rather harmless (since the conditions for --restrict-filenames should not change while a process is running), but just to be sure.
This also simplifies the interface for callers, who can just pass in the idiomatic None for "I don't care, whatever is the default".
11 years ago
4a98cdbf3b
YoutubeDL: set the 'params' property before any message/warning/error is sent ( fixes #1840 )
...
If it sets the 'restrictfilenames' param, it will first report a warning. It will try to get the logger from the 'params' property, which would be set at that moment to None, raising the error 'AttributeError: 'NoneType' object has no attribute 'get''
11 years ago
d31209a144
Use the 'extractor_key' field for the download archive file
...
It has the same value as the ie_key.
11 years ago
529a2e2cc3
Fix typo in the documentation of the 'download_archive' param
11 years ago
fb04e40396
[soundcloud] Support for listing of audio-only files
11 years ago
Jaime Marquínez Ferrándiz
Philipp Hagemeister
Sergey M․
codesparkle
Pierre
Andrei Troie
AndroKev
Mark Schreiber
cryptonaut
Jouke Waleson
George Boyle
Gjorgji Jankovski
rupertbaxter2
robbie
rzhxeo
alimirjamali