_ _ _ _ _ _ ___ _ _| |_ _ _| |__ ___ __| | |__ | | | |/ _ \| | | | __| | | | '_ \ / _ \_____ / _` | '_ \ | |_| | (_) | |_| | |_| |_| | |_) | __/_____| (_| | |_) | \__, |\___/ \__,_|\__|\__,_|_.__/ \___| \__,_|_.__/ |___/__ _ _ _ __ / /__ _ | | __ __ _ _ _ __| | |__\ \ | |/ _` | | |/ / / _` | | | | |/ _` | '_ \| | | | (_| |_| < | (_| |_ | |_| | (_| | |_) | | | |\__,_(_)_|\_(_)__,_(_) \__, |\__,_|_.__/| | \_\ |___/ /_/ Version: 2014aug05 This file: http://angg.twu.net/youtube-db/README.html See: http://angg.twu.net/ferramentas-para-ativistas.html http://angg.twu.net/ferramentas-para-ativistas.html#testes-javascript http://angg.twu.net/ferramentas-para-ativistas.html#testes-audio http://angg.twu.net/ferramentas-para-ativistas.html#testes-video What is this ============ This - ydb - is the software that I use to maintain the archive of videos at: http://angg.twu.net/linkdasruas2.html It can be used as a Lua library and as a toolbox, in arbitrarily complex ways; that is what I do at home to produce linkdasruas2.html from a very big database spread into several files. But what matters here is that ydb can also be used as a standalone script, invoked like this (details below!): ydb -simple myindex to produce a "myindex.html" from a "myindex.txt", possibly downloading local copies of the videos that the "myindex.txt" point to. The main idea ============= Take a look at "linkdasruas2.html". It is generated from a file "linkdasruas2.txt", that you can access here: http://angg.twu.net/linkdasruas2.txt.html http://angg.twu.net/linkdasruas2.txt The HTML contains in copy of that .txt in a
...
block, followed by a copy of this ".lst", http://angg.twu.net/linkdasruas2.lst.html http://angg.twu.net/linkdasruas2.lst For each line in the .txt with a link to youtube, e.g., http://www.youtube.com/watch?v=QJjOM7SpbhM J.S. Bach 6th Cello Suite, Courante the .lst contains the name of the "local copy" of that video, which can be something like: videos/J.S._Bach_6th_Cello_Suite_transcribed_for_viola_-_Courante-QJjOM7SpbhM.mp4 Note that the 11-char identifier, "QJjOM7SpbhM", is the same in the URL and in the MP4 file. All the modules in ydb use these identifiers to relate URLs to files. Dependencies ============ You will need lua5.1 and lua-posix. On Debian-based systems you can install them with just: sudo apt-get install lua5.1 lua-posix You will need a _very recent_ version of youtube-dl for downloading videos from Youtube. This page has installation instructions: http://rg3.github.io/youtube-dl/download.html You can do lots of tests and stuff without youtube-dl, though. This is optional but recommended: sudo apt-get install tkdiff Downloading, installing, testing ================================ Do this (the "{}"s are optional, but nice if you are a beginner pasting blocks of commands to a terminal): { rm -Rv ~/ydb/ mkdir ~/ydb/ cd ~/ydb/ wget http://angg.twu.net/youtube-db/ydb.tgz tar -xvzf ydb.tgz } { cp -v ~/ydb/test1.txt /tmp/ cd /tmp/ ~/ydb/ydb -fake -simple test1 source test1.sh ~/ydb/ydb -fake -simple test1 mv test1.txt2 test1.txt ~/ydb/ydb -fake -simple test1 } and then open this URL in a browser: file:///tmp/test1.html The output messages of each run of "~/ydb/ydb -simple ..." should be clear enough. In the first run, ydb detects which youtube URLs in the .txt don't have local mp4s, dates or titles; all these things can be downloaded from youtube with youtube-dl, and ydb produces a shell script, "test1.sh", that downloads all of them - but as we are using the "-fake" option ydb creates a script test1.sh that creates empty MP4 files and fake titles and dates instead of calling youtube-dl. The command ". test1.sh" runs the (fake) shell script, which "downloads" fake mp4s to /tmp/videos/, and stores fake dates and titles into /tmp/ydbtmp/. When we run ~/ydb/ydb -fake -simple test1 the second time, it finds the mp4s files in /tmp/videos/ and the "downloaded" dates and times in /tmp/ydbtmp/. It uses the names of these mp4s to produce a file test1.lst, and from test1.txt and test1.lst it produces a test1.html. It also creates a file test1.txt2 by adding the missing dates and titles to test1.txt. Normally we would compare test1.txt and test1.txt2 visually to be sure that the changes are ok, but in the script above we pretend that the they are indeed ok, and we simply run mv test1.txt2 test1.txt to override the old test1.txt with the new version. When we run ~/ydb/ydb -fake -simple test1 for the third time, it generates the .lst and the .html again, but now we know that the .lst has one .mp4 for each youtube URL in the .txt, and we also know that the .txt is "complete" in the sense that each URL line in it has a date and a title, so the .html is "complete" too. We browse the result, at: file:///tmp/test1.html and that's it. Cleaning up =========== Run: { rm -v /tmp/test1.* rm -Rv /tmp/videos/ rm -Rv /tmp/ydbtmp/ } A test without "-fake" ====================== After cleaning up, do: { cp -v ~/ydb/test1.txt /tmp/ cd /tmp/ ~/ydb/ydb -simple test1 . test1.sh ~/ydb/ydb -simple test1 mv test1.txt2 test1.txt ~/ydb/ydb -simple test1 } and browse: file:///tmp/test1.html Now change test1.txt - add URLs of videos that interest you, remove the ones that don't, etc, and run you-know-what to download local copies and to generate the .html (and the .txt2)... Cheers ====== The documentation ends abruptly here because this is the first public release. =) Eduardo Ochs eduardoochs@gmail.com https://www.facebook.com/eduardo.ochs http://angg.twu.net/ferramentas-para-ativistas.html#casos-particulares http://angg.twu.net/ferramentas-para-ativistas.html#audios http://angg.twu.net/ferramentas-para-ativistas.html#testes-javascript http://angg.twu.net/ferramentas-para-ativistas.html#testes-videos