Commit Graph

321 Commits (59b480128536601e90accfa036f24c58be187811)

Author SHA1 Message Date
Lesmiscore a904a7f8c6
Allow users to specify encoding in each config files ()
Authored by: Lesmiscore
pukkandan f5ea47488a
[cleanup] Minor fixes
pukkandan 54007a45f1
[cleanup] Consistent style for file heads
pukkandan ac66811112
[compat] Remove more functions
Removing any more will require changes to a large number of extractors
pukkandan 0f06bcd759
[cleanup] Minor fixes (See desc)
* [youtube] Fix `--youtube-skip-dash-manifest`
* [build] Use `$()` in `Makefile`. Closes 
* Fix bug in 385ffb467b
* Fix bug in 43d7f5a5d0
* [cleanup] Remove unnecessary `utf-8` from `str.encode`/`bytes.decode`
* [utils] LazyList: Expose unnecessarily "protected" attributes
and other minor cleanup
pukkandan 19a0394044
[cleanup] Misc cleanup and refactor ()
pukkandan f82711587c
[cleanup] Sort imports
Using https://github.com/PyCQA/isort

    isort -m VERTICAL_HANGING_INDENT --py 36 -l 80 --rr -n --tc .
pukkandan 86e5f3ed2e
[cleanup] Upgrade syntax
Using https://github.com/asottile/pyupgrade

1. `__future__` imports and `coding: utf-8` were removed
2. Files were rewritten with `pyupgrade --py36-plus --keep-percent-format`
3. f-strings were cherry-picked from `pyupgrade --py36-plus`

Extractors are left untouched (except removing header) to avoid unnecessary merge conflicts
felix cfb0511d82
[cleanup] Remove unused code paths ()
Notes:

* `_windows_write_string`: Fixed in 3.6
  * https://bugs.python.org/issue1602
  * PEP: https://www.python.org/dev/peps/pep-0528

* Windows UTF-8 fix: Fixed in 3.3
  * https://bugs.python.org/issue13216

* `__loader__`: is always present in 3.3+
  * https://bugs.python.org/issue14646

* `workaround_optparse_bug9161`: Fixed in 2.7
  * https://bugs.python.org/issue9161

Authored by: fstirlitz
pukkandan b506289fe2
[test] Add `test_locked_file`
coletdev 1c1b2f96ae
[youtube:tab] Fix duration extraction for shorts ()
Related: https://github.com/TeamNewPipe/NewPipe/issues/8034
Authored-by: coletdjnz
pukkandan 5c3895fff1
[outtmpl] Limit changes during sanitization
Closes 
s0u1h eeb2a770f3
[utils] `format_decimal_suffix`: Fix for very large numbers ()
Authored by: s0u1h
pukkandan 28469edd7d
Release 2022.02.03
pukkandan 8bd1c00bf3
[utils] Handle `ss:xxx` in `parse_duration`
Closes 
Zenon Mousmoulas 0254f16274
[utils] Improve `get_elements_text_and_html_by_attribute` regex ()
Authored by: zmousm, pukkandan
Zenon Mousmoulas 6f32a0b5b7
[utils] Improve parsing for nested HTML elements ()
and add functions to return the HTML of elements

Authored by: zmousm
pukkandan 06e57990f7
Allow multiple and nested configuration files
Pierre Mdawar f02d24d8d2
[utils] Fix `format_bytes` output for Bytes ()
Authored by: pukkandan, mdawar
pukkandan 352d5da812
[utils] Improve `parse_count`
pukkandan 282f570918
[utils] Fix error when copying `LazyList`
pukkandan c5e3f84972
[utils] Allow alignment in `render_table`
and add tests
Damiano Amatruda 17ec8bcfa9
[microsoftstream] Add extractor ()
Based on: https://github.com/ytdl-org/youtube-dl/pull/24649
Fixes: https://github.com/ytdl-org/youtube-dl/issues/24440
Authored by: damianoamatruda, nixklai
pukkandan aa7785f860
[utils] Standardize timestamp formatting code
Closes 
pukkandan 176f1866cb
Add HDR information to formats
pukkandan 18f96d129b
[utils] Allow duration strings in filter
Closes 
pukkandan 4dfbf8696b
[utils] Add `parse_qs`
pukkandan 8f18aca871
Let `--match-filter` reject entries early
Makes redundant: `--match-title`, `--reject-title`, `--min-views`, `--max-views`
pukkandan a047eeb6d2
Add regex to `--match-filter`
This does not fully deprecate `--match-title`/`--reject-title`
since `--match-filter` is only checked after the extraction is complete,
while `--match-title` can often be checked from the flat playlist.

Fixes: https://github.com/ytdl-org/youtube-dl/issues/9092, https://github.com/ytdl-org/youtube-dl/issues/23035
Max Teegen 77b87f0519 Add all format filtering operators also to `--match-filter`
PR: https://github.com/ytdl-org/youtube-dl/pull/27361

Authored by: max-te
pukkandan e0f2b4b47d
[utils] Fix slicing of reversed `LazyList`
Closes 
felix c843e68588
[utils] Improve `js_to_json` comment regex
Capture the newline character as part of a single-line comment

From , Authored by: fstirlitz
pukkandan 981052c9c6 Some minor fixes and refactoring (see desc)
* [utils] Fix issues with reversal
* check_formats should catch `DownloadError`, not `ExtractorError`
* Simplify format selectors with `LazyList` and `yield from`
pukkandan 28419ca2c8
[utils] Improve `LazyList`
* Add `repr` and `str` that mimics `list`
* Add `reversed`. Unlike `[::-1]`, reversed does not exhaust the iterable and modifies the `LazyList` in-place
* Add tests
pukkandan 639f1cea92
Fix `%d` and empty default in outtmpl
Closes 
felix cc52de4356
[cleanup] Point all shebang to `python3` ()
Authored by: fstirlitz
pukkandan bc6b9bcd65
[utils] Escape URLs in `sanitized_Request`, not `sanitize_url`
d2558234cf added escaping of URLs while sanitizing. However, `sanitize_url` may not always receive an actual URL.
Eg: When using `yt-dlp "search query" --default-search ytsearch`, `search query` gets escaped to `search%20query` before being prefixed with `ytsearch:` which is not the intended behavior. So the escaping is moved to `sanitized_Request` instead.
Hubert Hirtz 5435dcf96e
Handle Basic Auth `user:pass` in URLs
Fixes https://github.com/ytdl-org/youtube-dl/issues/20258, https://github.com/ytdl-org/youtube-dl/issues/26211
Authored by: hhirtz, pukkandan
colethedj 9e62f283ff
[utils] Add `datetime_from_str` to parse relative time ()
and `datetime_add_months` to accurately add/subtract months

Authored by: colethedj
Pccode66 7a5c1cfe93
Completely change project name to yt-dlp ()
* All modules and binary names are changed
* All documentation references changed
* yt-dlp no longer loads youtube-dlc config files
* All URLs changed to point to organization account

Co-authored-by: Pccode66
Co-authored-by: pukkandan
pukkandan 00dd0cd573 Update to ytdl-2021.01.08
pukkandan 29f7c58aaf Update to ytdl-2021.01.03
pukkandan 732044afb2 Add --write-*-link by h-h-h-h
Authored-by: h-h-h-h
pukkandan 8bdd16b499 Merge 'ytdl-org/youtube-dl/master' release 2020.11.19
Old Extractors left behind:
	VLivePlaylistIE
	YoutubeSearchURLIE
	YoutubeShowIE
	YoutubeFavouritesIE

If removing old extractors, make corresponding changes in
	docs/supportedsites.md
	youtube_dlc/extractor/extractors.py

Not merged:
	.github/ISSUE_TEMPLATE/1_broken_site.md
	.github/ISSUE_TEMPLATE/2_site_support_request.md
	.github/ISSUE_TEMPLATE/3_site_feature_request.md
	.github/ISSUE_TEMPLATE/4_bug_report.md
	.github/ISSUE_TEMPLATE/5_feature_request.md
	test/test_all_urls.py
	youtube_dlc/version.py
	Changelog
Unknown 39e7107d7f Merge branch 'ytdl-org-master'
Unknown 8ef153ee6f rot47 capital letters.
Unknown 3867038a06 renaming issues resolved
Unknown cefecac12c [skip travis] renaming
to avoid using same folder when using pip install for example
Sergey M․ 42db58ec73
[utils] Improve str_to_int
Remita Amine 348c6bf1c1 [utils] handle int values passed to str_to_int
Sergey M․ 1ced222120
[utils] Add generic caesar cipher and rot47
InfernalUnderling 9d30c2132a [utils] Handle rd-suffixed day parts in unified_strdate ()
Sergey M․ 824fa51165
[utils] Improve subtitles_filename (closes )
Sergey M․ 28cc2241e4
[utils] Restrict parse_codecs and add theora as known vcodec ()
Sergey M․ 53cd37bac5
[utils] Improve strip_or_none
Jakub Wilk fd35d8cdfd [utils] Transliterate "þ" as "th" ()
Despite visual similarity "þ" is unrelated to "p".
It is normally transliterated as "th":

    $ echo þ-Þ | iconv -t ASCII//TRANSLIT
    th-TH
Sergey M․ 5e1271c56d
[utils] Improve int_or_none and float_or_none ()
Sergey M․ 0dc41787af
[utils] Introduce parse_bitrate
Sergey M․ fad4ceb534
[utils] Fix urljoin for paths with non-http(s) schemes
Sergey M․ 25d110be30
[utils] Properly recognize AV1 codec (closes )
Sergey M․ af03000ad5
[utils] Introduce url_or_none
Sergey M․ e9c671d5e8
[utils] Allow JSONP with empty func name (closes )
Enes 85750f8972 [openload] Improve ext extraction
Remita Amine 3bb3ff38a1 [test_utils] add tests for b836118724
Sergey M․ 6cc622327f
[utils] Introduce merge_dicts
Sergey M․ 1cc47c6674
[utils] Fix match_str for boolean meta fields
Philipp Hagemeister f226880c6d [tennistv] Add support for tennistv.com
Sergey M․ b871d7e954
[utils] Add parse_resolution
Sergey M․ befa4708fd
[utils] Fixup some common URL's typos in sanitize_url (closes )
Sergey M․ c707b1d828
[test_utils] Add tests for malformed JSON handling in js_to_json
Mike Fährmann c384d537f8 [util] Improve scientific notation handling in js_to_json (closes )
Sergey M․ b555ae9bf1
[utils] Add another date format pattern ()
Sergey M․ 056653bbb1
[utils] Add support for zero years and months in parse_duration
Yen Chi Hsuan 3869028ffb [utils] Use bytes-like objects in dfxp2srt
This fixes handling of non-UTF8 TTML subtitles

Closes 
Yen Chi Hsuan 95f3f7c20a
[utils] Fix unescapeHTML for misformed string like "&a"" ()
Sergey M․ 5b232f46dc
[utils] Skip missing params in cli_bool_option (closes )
Sergey M․ dee2ff1d81
[test_utils] Fix tests under Windows
Yen Chi Hsuan 609ff8ca19 [utils] Support attributes with no values in get_elements_by_attribute()
Sergey M․ b4a3d461e4
[utils] Handle HTMLParseError in extract_attributes (closes )
Sergey M․ 2ae2ffda5e
[utils] Improve unified_timestamp
Yen Chi Hsuan 5552c9eb0f
[utils] Recognize more patterns in strip_jsonp()
Used in Youku Show pages
Yen Chi Hsuan 0c26548601
[cda] Implement birthday verification (closes )
Sergey M․ deef31955b
[utils] Improve unified_timestamp
Seen at http://zaq1.pl/video/xev0e
Tithen-Firion 9222d94510 [test_utils] Add one more clean_html test
Remita Amine 5b995f713b [utils] add support for ttml styles
Sergey M․ a426ef6d78
[test_utils] Do not use dash in env variables' names
Sergey M․ 41c5e60dd5
[test_utils] Fix expand_path tests
Sergey M․ 51098426b8
[utils] Introduce expand_path
Sergey M․ 4b5de77bdb
[utils] Process bytestrings in urljoin (closes )
Yen Chi Hsuan f48409c7ac [utils] Add pkcs1pad
Used in daisuki.net ()
Thomas Christlieb 2af12ad9d2 Introduce get_elements_by_class and get_elements_by_attribute utility functions
Sergey M․ 4195096ea8
[utils] Improve comments processing in js_to_json (closes )
Michal Čihař b3ee552e4b
[utils] Handle single-line comments in js_to_json
Sergey M․ 15846398ca
[utils] Improve parse_duration
Sergey M․ cb655f34fb
[utils] Add more date formats
Remita Amine 7fe1592073 [common] fix dash codec information for mixed videos and fragment url construction()
Sergey M․ b0c65c677f
[utils] Improve urljoin
Sergey M․ e34c33614d
[utils] Add convenience urljoin
Yen Chi Hsuan 582be35847
Update coding style after pycodestyle 2.1.0
In pycodestyle 2.1.0, E305 was introduced, which requires two blank
lines after top level declarations, too.

See https://github.com/PyCQA/pycodestyle/issues/400

See also ; thanks @stepshal for first mentioning this issue and
initial patches
Sergey M․ 02dc0a36b7
[utils] Introduce base_url