Browser Extension for Tube Archivist
+
+
+
+
+
+## Core Functionality
+This is a browser extension to bridge YouTube with [Tube Archivist](https://github.com/tubearchivist/tubearchivist), your self hosted YouTube media server.
+- Add your Tube Archivist connection details in the addon popup.
+- On YouTube video pages, inject a download button to download that video and a subscribe button to subscribe to that channel.
+- On YouTube channel pages, inject a button to subscribe to the channel or download the complete channel. Regarding the channel subpages, this follows the same rules as adding to the queue over the form.
+- Throughout most places, hover over the video title to reveal a download button for that video.
+- Sync your cookies for yt-dlp.
+
+## Screenshots
+
+Popup to enter your connection details.
+
+
++ + +Button injected on video page to download the video or subscribe to the channel. +
+ + +Download button injected showing when hovering over the video title. +
+ + +Channel button injected to subscribe or download whole channel, video download button showing when hovering over the video title. +
+ +## Install +- Firefox: The addon is available on the [Extension store](https://addons.mozilla.org/addon/tubearchivist-companion/). +- Chrome: The addon is available on the [Chrome Web Store](https://chrome.google.com/webstore/detail/tubearchivist-companion/jjnkmicfnfojkkgobdfeieblocadmcie). + +## Update +After a new release here on GitHub, you'll get updates automatically in your browser. Due to the verification process, for Firefox this usually takes 1-2 hours, for Chrome 2-3 days. + +## Permissions +- **Access your data for www.youtube.com**: Needed to inject download and subscribe buttons directly into the page. +- **Storage**: Needed to store your connection details. +- **Cookie**: Needed to read your cookies for youtube.com to access restricted videos. + +## Setup +- **URL**: This is where your Tube Archivist instance is located. Can be a host name or an IP address. Add the port if needed at the end, e.g. `:8000`. +- **API key**: You can find your API key on the settings page (Settings -> Application -> Integrations section -> API token) of your Tube Archivist instance. + +A green checkmark will appear next to the *Save* button if your connection is working. + +## Options +- **Continuous Cookie Sync**: Automatically and continuously update the cookie on change. +- **Copy Now**: Copy the cookie now to TA. +- **Show Cookie**: Show the cookie on click, for copy paste. +- **Autostart**: Autostart and prioritize videos send from this extension. + +## Test this extension +Before continuing loading the temporary extension here, make sure to deactivate/delete the main extension first. + +Symlink/copy the correct manifest file for your browser to the expected location, e.g. `ln -s manifest-firefox.json manifest.json`. + +- Firefox + - Open `about:debugging#/runtime/this-firefox` + - Click on *Load Temporary Add-on* + - Select the *manifest.json* file to load the addon. + - You can *inspect* background.js by lunching the debug tools from there. +- Chrome / Chromium + - Open `chrome://extensions/` + - Toggle *Developer mode* on top right + - Click on *Load unpacked* + - Open the folder containing the *manifest.json* file. + - Click on *Service Worker* to open the dev tools at background.js. + +Note: +- If you are running your TA dev setup outside of the container, you need to point the URL to the backend and _not_ the frontend. E.g. localhost:8000 and not localhost:3000. + +## Compatibility +- Verify that you are running the [latest version](https://github.com/tubearchivist/tubearchivist/releases/latest) of Tube Archivist as the API is under development and will change. +- For testing this extension between releases, use the *unstable* builds of Tube Archivist, only for your testing environment. + +## Roadmap +Join us on [Discord](https://www.tubearchivist.com/discord) and help us improve and extend this project. This is a list of planned features, in no particular order: +- [ ] Implement download/subscribe button for playlists +- [ ] Add download buttons to the `/shorts/` pages +- [X] Get download and subscribe status from TA to show on the injected buttons +- [X] Implement download button for videos on the YouTube homepage over inline preview +- [X] Implement download button for videos on playlist +- [X] Error handling for connection errors +- [X] Dynamically inject buttons with mutation observer + +## Making changes to the JavaScript +The JavaScript does not require any build step; you just edit the files directly. However, there is config for eslint and prettier (a linter and formatter respectively); their use is recommended but not required. To use them, install `node`, run `npm i` from the root directory of this repository to install dependencies, then run `npm run lint` and `npm run format` to run eslint and prettier respectively. + +## Updating Artwork +Google listing is *very* picky. Screenshots need to be exactly **1280x800** in resolution and need to be in *jpg* or *png* without alpha canal. diff --git a/browser-extension/browser-extension-0.4.2/assets/icon-128x128.png b/browser-extension/browser-extension-0.4.2/assets/icon-128x128.png new file mode 100644 index 0000000..cefd1ee Binary files /dev/null and b/browser-extension/browser-extension-0.4.2/assets/icon-128x128.png differ diff --git a/browser-extension/browser-extension-0.4.2/assets/screenshot-video.png b/browser-extension/browser-extension-0.4.2/assets/screenshot-video.png new file mode 100644 index 0000000..b6bcfb5 Binary files /dev/null and b/browser-extension/browser-extension-0.4.2/assets/screenshot-video.png differ diff --git a/browser-extension/browser-extension-0.4.2/assets/social-preview-1400x560.png b/browser-extension/browser-extension-0.4.2/assets/social-preview-1400x560.png new file mode 100644 index 0000000..b1be592 Binary files /dev/null and b/browser-extension/browser-extension-0.4.2/assets/social-preview-1400x560.png differ diff --git a/browser-extension/browser-extension-0.4.2/assets/social-preview-440x280.png b/browser-extension/browser-extension-0.4.2/assets/social-preview-440x280.png new file mode 100644 index 0000000..87187e0 Binary files /dev/null and b/browser-extension/browser-extension-0.4.2/assets/social-preview-440x280.png differ diff --git a/browser-extension/browser-extension-0.4.2/assets/social-preview-920x680.png b/browser-extension/browser-extension-0.4.2/assets/social-preview-920x680.png new file mode 100644 index 0000000..c18f1ef Binary files /dev/null and b/browser-extension/browser-extension-0.4.2/assets/social-preview-920x680.png differ diff --git a/browser-extension/browser-extension-0.4.2/assets/social-preview.png b/browser-extension/browser-extension-0.4.2/assets/social-preview.png new file mode 100644 index 0000000..91fb382 Binary files /dev/null and b/browser-extension/browser-extension-0.4.2/assets/social-preview.png differ diff --git a/browser-extension/browser-extension-0.4.2/assets/tac-screenshot-channel.jpg b/browser-extension/browser-extension-0.4.2/assets/tac-screenshot-channel.jpg new file mode 100644 index 0000000..738f2ea Binary files /dev/null and b/browser-extension/browser-extension-0.4.2/assets/tac-screenshot-channel.jpg differ diff --git a/browser-extension/browser-extension-0.4.2/assets/tac-screenshot-search.jpg b/browser-extension/browser-extension-0.4.2/assets/tac-screenshot-search.jpg new file mode 100644 index 0000000..fce073d Binary files /dev/null and b/browser-extension/browser-extension-0.4.2/assets/tac-screenshot-search.jpg differ diff --git a/browser-extension/browser-extension-0.4.2/assets/tac-screenshot.png b/browser-extension/browser-extension-0.4.2/assets/tac-screenshot.png new file mode 100644 index 0000000..637e427 Binary files /dev/null and b/browser-extension/browser-extension-0.4.2/assets/tac-screenshot.png differ diff --git a/browser-extension/browser-extension-0.4.2/assets/tube-archivist-companion-banner.png b/browser-extension/browser-extension-0.4.2/assets/tube-archivist-companion-banner.png new file mode 100644 index 0000000..564e7d5 Binary files /dev/null and b/browser-extension/browser-extension-0.4.2/assets/tube-archivist-companion-banner.png differ diff --git a/browser-extension/browser-extension-0.4.2/deploy.sh b/browser-extension/browser-extension-0.4.2/deploy.sh new file mode 100755 index 0000000..b8896a7 --- /dev/null +++ b/browser-extension/browser-extension-0.4.2/deploy.sh @@ -0,0 +1,54 @@ +#!/bin/bash +# build package + +set -e + +if [[ $(basename "$(pwd)") != 'browser-extension' ]]; then + echo 'not in browser-extension folder' + exit 1 +fi + +echo "latest tags:" +git tag | tail -n 5 | sort -r + +printf "\ncreate new version:\n" +read -r VERSION + + +# build release zip files +function create_zip { + cd extension + + # firefox + rm manifest.json + cp manifest-firefox.json manifest.json + zip -rq ../release/ta-companion-"$VERSION"-firefox.zip . \ + -x manifest-chrome.json -x manifest-firefox.json + + # chrome + rm manifest.json + cp manifest-chrome.json manifest.json + zip -rq ../release/ta-companion-"$VERSION"-chrome.zip . \ + -x manifest-chrome.json -x manifest-firefox.json + + rm manifest.json + cd .. + +} + + +# create release tag +function create_release { + + git tag -a "$VERSION" -m "new release version $VERSION" + git push origin "$VERSION" + +} + + +create_zip +create_release + + +## +exit 0 diff --git a/browser-extension/browser-extension-0.4.2/extension/background.js b/browser-extension/browser-extension-0.4.2/extension/background.js new file mode 100644 index 0000000..6907458 --- /dev/null +++ b/browser-extension/browser-extension-0.4.2/extension/background.js @@ -0,0 +1,321 @@ +/* +extension background script listening for events +*/ + +'use strict'; + +console.log('running background.js'); + +let browserType = getBrowser(); + +// boilerplate to dedect browser type api +function getBrowser() { + if (typeof chrome !== 'undefined') { + if (typeof browser !== 'undefined') { + return browser; + } else { + return chrome; + } + } else { + console.log('failed to detect browser'); + throw 'browser detection error'; + } +} + +// send get request to API backend +async function sendGet(path) { + let access = await getAccess(); + const url = `${access.url}:${access.port}/${path}`; + console.log('GET: ' + url); + + const rawResponse = await fetch(url, { + method: 'GET', + headers: { + Accept: 'application/json', + 'Content-Type': 'application/json', + Authorization: 'Token ' + access.apiKey, + mode: 'no-cors', + }, + }); + + const content = await rawResponse.json(); + return content; +} + +// send post/put request to API backend +async function sendData(path, payload, method) { + let access = await getAccess(); + const url = `${access.url}:${access.port}/${path}`; + console.log(`${method}: ${url}`); + if (!path.endsWith('cookie/')) console.log(`${method}: ${JSON.stringify(payload)}`); + + try { + const rawResponse = await fetch(url, { + method: method, + headers: { + Accept: 'application/json', + 'Content-Type': 'application/json', + Authorization: 'Token ' + access.apiKey, + mode: 'no-cors', + }, + body: JSON.stringify(payload), + }); + const content = await rawResponse.json(); + return content; + } catch (e) { + console.error(e); + return null; + } +} + +// read access details from storage.local +async function getAccess() { + let storage = await browserType.storage.local.get('access'); + + return storage.access; +} + +// check if cookie is valid +async function getCookieState() { + const path = 'api/appsettings/cookie/'; + let response = await sendGet(path); + console.log('cookie state: ' + JSON.stringify(response)); + + return response; +} + +// send ping to server +async function verifyConnection() { + const path = 'api/ping/'; + let message = await sendGet(path); + console.log('verify connection: ' + JSON.stringify(message)); + + if (message?.response === 'pong') { + return true; + } else if (message?.detail) { + throw new Error(message.detail); + } else { + throw new Error(`got unknown message ${JSON.stringify(message)}`); + } +} + +// send youtube link from injected buttons +async function download(url) { + let apiURL = 'api/download/'; + let autostart = await browserType.storage.local.get('autostart'); + if (Object.keys(autostart).length > 0 && autostart.autostart.checked) { + apiURL += '?autostart=true'; + } + return await sendData( + apiURL, + { + data: [ + { + youtube_id: url, + status: 'pending', + }, + ], + }, + 'POST' + ); +} + +async function subscribe(url, subscribed) { + return await sendData( + 'api/channel/', + { + data: [ + { + channel_id: url, + channel_subscribed: subscribed, + }, + ], + }, + 'POST' + ); +} + +async function videoExists(id) { + const path = `api/video/${id}/`; + let response = await sendGet(path); + if (response?.error) return false; + let access = await getAccess(); + return new URL(`video/${id}/`, `${access.url}:${access.port}/`).href; +} + +async function getChannel(channelHandle) { + const path = `api/channel/search/?q=${channelHandle}`; + try { + return await sendGet(path); + } catch { + return false; + } +} + +async function cookieStr(cookieLines) { + const path = 'api/appsettings/cookie/'; + let payload = { + cookie: cookieLines.join('\n'), + }; + let response = await sendData(path, payload, 'PUT'); + + return response; +} + +function buildCookieLine(cookie) { + // 2nd argument controls subdomains, and must match leading dot in domain + let includeSubdomains = cookie.domain.startsWith('.') ? 'TRUE' : 'FALSE'; + + return [ + cookie.domain, + includeSubdomains, + cookie.path, + cookie.httpOnly.toString().toUpperCase(), + Math.trunc(cookie.expirationDate) || 0, + cookie.name, + cookie.value, + ].join('\t'); +} + +async function getCookieLines() { + const acceptableDomains = ['.youtube.com', 'youtube.com', 'www.youtube.com']; + let cookieStores = await browserType.cookies.getAllCookieStores(); + let cookieLines = [ + '# Netscape HTTP Cookie File', + '# https://curl.haxx.se/rfc/cookie_spec.html', + '# This is a generated file! Do not edit.\n', + ]; + for (let i = 0; i < cookieStores.length; i++) { + const cookieStore = cookieStores[i]; + let allCookiesStore = await browserType.cookies.getAll({ + domain: '.youtube.com', + storeId: cookieStore['id'], + }); + for (let j = 0; j < allCookiesStore.length; j++) { + const cookie = allCookiesStore[j]; + if (acceptableDomains.includes(cookie.domain)) { + cookieLines.push(buildCookieLine(cookie)); + } + } + } + return cookieLines; +} + +async function sendCookies() { + console.log('function sendCookies'); + let cookieLines = await getCookieLines(); + let response = cookieStr(cookieLines); + + return response; +} + +let listenerEnabled = false; +let isThrottled = false; + +async function handleContinuousCookie(checked) { + if (checked === true) { + browserType.cookies.onChanged.addListener(onCookieChange); + listenerEnabled = true; + console.log('Cookie listener enabled'); + } else { + browserType.cookies.onChanged.removeListener(onCookieChange); + listenerEnabled = false; + console.log('Cookie listener disabled'); + } +} + +function onCookieChange(changeInfo) { + if (!isThrottled) { + isThrottled = true; + + console.log('Cookie event detected:', changeInfo); + + sendCookies(); + + setTimeout(() => { + isThrottled = false; + }, 10000); + } +} + +/* +process and return message if needed +the following messages are supported: +type Message = + | { type: 'verify' } + | { type: 'cookieState' } + | { type: 'sendCookie' } + | { type: 'getCookieLines' } + | { type: 'continuousSync', checked: boolean } + | { type: 'download', url: string } + | { type: 'subscribe', url: string } + | { type: 'unsubscribe', url: string } + | { type: 'videoExists', id: string } + | { type: 'getChannel', url: string } +*/ +function handleMessage(request, sender, sendResponse) { + console.log('message background.js listener got message', request); + + // this function must return the value `true` in chrome to signal the response will be async; + // it cannot return a promise + // so in order to use async/await, we need a wrapper + (async () => { + switch (request.type) { + case 'verify': { + return await verifyConnection(); + } + case 'cookieState': { + return await getCookieState(); + } + case 'sendCookie': { + return await sendCookies(); + } + case 'getCookieLines': { + return await getCookieLines(); + } + case 'continuousSync': { + return await handleContinuousCookie(request.checked); + } + case 'download': { + return await download(request.url); + } + case 'subscribe': { + return await subscribe(request.url, true); + } + case 'unsubscribe': { + let channel = await getChannel(request.url); + return await subscribe(channel.channel_id, false); + } + case 'videoExists': { + return await videoExists(request.videoId); + } + case 'getChannel': { + return await getChannel(request.channelHandle); + } + default: { + let err = new Error(`unknown message type ${JSON.stringify(request.type)}`); + console.log(err); + throw err; + } + } + })() + .then(value => sendResponse({ success: true, value })) + .catch(e => { + console.log(e); + let message = e?.message ?? e; + if (message === 'Failed to fetch') { + // chrome's error message for failed `fetch` is not very user-friendly + message = 'Could not connect to server'; + } + sendResponse({ success: false, value: message }); + }); + return true; +} + +browserType.runtime.onMessage.addListener(handleMessage); + +browserType.runtime.onStartup.addListener(() => { + browserType.storage.local.get('continuousSync', data => { + handleContinuousCookie(data?.continuousSync?.checked || false); + }); +}); diff --git a/browser-extension/browser-extension-0.4.2/extension/images/icon.png b/browser-extension/browser-extension-0.4.2/extension/images/icon.png new file mode 100644 index 0000000..84afcbf Binary files /dev/null and b/browser-extension/browser-extension-0.4.2/extension/images/icon.png differ diff --git a/browser-extension/browser-extension-0.4.2/extension/images/icon128.png b/browser-extension/browser-extension-0.4.2/extension/images/icon128.png new file mode 100644 index 0000000..e96aba9 Binary files /dev/null and b/browser-extension/browser-extension-0.4.2/extension/images/icon128.png differ diff --git a/browser-extension/browser-extension-0.4.2/extension/images/icon16.png b/browser-extension/browser-extension-0.4.2/extension/images/icon16.png new file mode 100644 index 0000000..156cfff Binary files /dev/null and b/browser-extension/browser-extension-0.4.2/extension/images/icon16.png differ diff --git a/browser-extension/browser-extension-0.4.2/extension/images/logo.png b/browser-extension/browser-extension-0.4.2/extension/images/logo.png new file mode 100644 index 0000000..c7ecd8e Binary files /dev/null and b/browser-extension/browser-extension-0.4.2/extension/images/logo.png differ diff --git a/browser-extension/browser-extension-0.4.2/extension/images/question.svg b/browser-extension/browser-extension-0.4.2/extension/images/question.svg new file mode 100644 index 0000000..872deef --- /dev/null +++ b/browser-extension/browser-extension-0.4.2/extension/images/question.svg @@ -0,0 +1,16 @@ + + + diff --git a/browser-extension/browser-extension-0.4.2/extension/images/social/discord.svg b/browser-extension/browser-extension-0.4.2/extension/images/social/discord.svg new file mode 100644 index 0000000..dd07931 --- /dev/null +++ b/browser-extension/browser-extension-0.4.2/extension/images/social/discord.svg @@ -0,0 +1,24 @@ + + + diff --git a/browser-extension/browser-extension-0.4.2/extension/images/social/github.svg b/browser-extension/browser-extension-0.4.2/extension/images/social/github.svg new file mode 100644 index 0000000..c693abf --- /dev/null +++ b/browser-extension/browser-extension-0.4.2/extension/images/social/github.svg @@ -0,0 +1,23 @@ + + + diff --git a/browser-extension/browser-extension-0.4.2/extension/images/social/reddit.svg b/browser-extension/browser-extension-0.4.2/extension/images/social/reddit.svg new file mode 100644 index 0000000..e6c615c --- /dev/null +++ b/browser-extension/browser-extension-0.4.2/extension/images/social/reddit.svg @@ -0,0 +1,35 @@ + + + diff --git a/browser-extension/browser-extension-0.4.2/extension/index.html b/browser-extension/browser-extension-0.4.2/extension/index.html new file mode 100644 index 0000000..a689e6b --- /dev/null +++ b/browser-extension/browser-extension-0.4.2/extension/index.html @@ -0,0 +1,71 @@ + + + + + + +
+ 
+ 
+
+
+ (showRecovery = false)} />
+ {/if}
+
diff --git a/ui/src/components/DashboardControls.svelte b/ui/src/components/DashboardControls.svelte
new file mode 100644
index 0000000..c4d4088
--- /dev/null
+++ b/ui/src/components/DashboardControls.svelte
@@ -0,0 +1,106 @@
+
+
+
+
+
+
+
+
+ {#if showRecovery}
+
+
+
+
+
+ (showRecovery = true)}
+ />
+
+
+ + Control Deck +
+
+
+
+
+
+
+
+
diff --git a/ui/src/components/LogViewer.svelte b/ui/src/components/LogViewer.svelte
new file mode 100644
index 0000000..cd4f8a5
--- /dev/null
+++ b/ui/src/components/LogViewer.svelte
@@ -0,0 +1,92 @@
+
+
+
+
+
+
+
+
+ {#if orphanResult}
+
+ {orphanResult}
+
+ {/if}
+
+
diff --git a/ui/src/components/RecoveryModal.svelte b/ui/src/components/RecoveryModal.svelte
new file mode 100644
index 0000000..e703d2d
--- /dev/null
+++ b/ui/src/components/RecoveryModal.svelte
@@ -0,0 +1,231 @@
+
+
+
+
+
+ {title}
+
+
+
+
+
+
diff --git a/ui/src/components/StatsCard.svelte b/ui/src/components/StatsCard.svelte
new file mode 100644
index 0000000..ede5a62
--- /dev/null
+++ b/ui/src/components/StatsCard.svelte
@@ -0,0 +1,39 @@
+
+
+
+
+
+
+
+
+
+ + Advanced Recovery +
+ +
+
+
+
+
+ Status: {status}
+
+ {#each ["unindexed", "rescue", "redundant", "lost"] as tab}
+
+ {/each}
+
+
+
+
+ {#if scanning && (!results[activeTab] || results[activeTab].length === 0)}
+
+
+
+ {/if}
+
+
+ Scanning...
+
+ {:else}
+ | Video ID | +Filename / Path | +Size/Info | +Action | +
|---|---|---|---|
| {item.video_id} | +{item.filename || item.path} | +{item.size_mb + ? item.size_mb + " MB" + : item.ta_source || "-"} | ++ {#if activeTab === "unindexed"} + + {:else if activeTab === "redundant"} + + {:else if activeTab === "lost"} + + + {:else} + + {/if} + | +
| No items found. | |||
+
diff --git a/ui/src/components/StatsManager.svelte b/ui/src/components/StatsManager.svelte
new file mode 100644
index 0000000..03bea49
--- /dev/null
+++ b/ui/src/components/StatsManager.svelte
@@ -0,0 +1,82 @@
+
+
+
+
+
+
+ {title}
+
+ {value}
+
+
+
+
+
+
diff --git a/ui/src/components/TranscodeManager.svelte b/ui/src/components/TranscodeManager.svelte
new file mode 100644
index 0000000..2e06a85
--- /dev/null
+++ b/ui/src/components/TranscodeManager.svelte
@@ -0,0 +1,188 @@
+
+
+
+
+
diff --git a/ui/src/components/VideoTable.svelte b/ui/src/components/VideoTable.svelte
new file mode 100644
index 0000000..080baac
--- /dev/null
+++ b/ui/src/components/VideoTable.svelte
@@ -0,0 +1,134 @@
+
+
+
+
+
+
+
+ + Transcode Queue +
++ Found {total} videos requiring transcode. +
+
+
+
+
+
+
+
+
+
+
+
+ {#if pages > 1}
+
+
+
+ Page {page} of {pages}
+
+
+ {/if}
+
+
+
+
diff --git a/ui/src/layouts/Layout.astro b/ui/src/layouts/Layout.astro
new file mode 100644
index 0000000..7f6503f
--- /dev/null
+++ b/ui/src/layouts/Layout.astro
@@ -0,0 +1,99 @@
+---
+import "../styles/global.css";
+
+interface Props {
+ title: string;
+}
+
+const { title } = Astro.props;
+---
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ + Video Matrix +
+ +
+
+
+
+
+
+
+
+
+
+
+ Showing {filteredVideos.length} / {videos.length} videos
+
+
+
+
+
+
+