website/public/blog/selfhost-search-engine/index.html

36 lines
17 KiB
HTML
Raw Normal View History

2024-03-13 21:22:06 +00:00
<!doctype html><html lang=en><head><meta charset=utf-8><meta content="width=device-width,initial-scale=1.0" name=viewport><meta content="light dark" name=color-scheme><title>Host your own private search engine with SearXNG</title><link href=/img/favicon-32x32.png rel=icon sizes=32x32 type=image/png><link href=/img/favicon-16x16.png rel=icon sizes=16x16 type=image/png><link href=/img/apple-touch-icon.png rel=apple-touch-icon sizes=180x180><link href=https://fonts.googleapis.com rel=preconnect><link crossorigin href=https://fonts.gstatic.com rel=preconnect><link href="https://fonts.googleapis.com/css2?family=Signika&display=swap" rel=stylesheet><style>*{font-family:monospace!important}body{--primary-color:#8070c6;--primary-pale-color:#8070c61c;--text-color:#151517;--text-pale-color:#454449;--bg-color:#ece5ea;--highlight-mark-color:#5f75b045;--callout-note-color:#e887bb;--callout-important-color:#a292e8;--callout-warning-color:#d9d564;--callout-alert-color:#f06969;--callout-question-color:#78b9c4;--callout-tip-color:#91d65c}body.dark{--primary-color:#a292e8;--primary-pale-color:#a292e81c;--text-color:#ece5ea;--text-pale-color:#5c5c61;--bg-color:#151517;--highlight-mark-color:#5f75b045;--callout-note-color:#e887bb;--callout-important-color:#a292e8;--callout-warning-color:#d9d564;--callout-alert-color:#f06969;--callout-question-color:#78b9c4;--callout-tip-color:#91d65c}body{--main-font:'Signika',ui-sans-serif,system-ui,-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,"Helvetica Neue",Arial,"Noto Sans",sans-serif,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol","Noto Color Emoji";--code-font:ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,"Liberation Mono","Courier New",monospace;--homepage-max-width:750px;--main-max-width:750px;--avatar-size:70px;--paragraph-font-size:18px;--paragraph-line-height:1.75;--aside-font-size:16px;--img-border-radius:0;--inline-code-border-radius:2px}</style><link href=/main.css rel=stylesheet><body class=post><script>if(localStorage.getItem('theme')=='dark'){document.body.classList.add('dark');const a=document.querySelector('link#hl');if(a)a.href='/hl-dark.css'}</script><header class=blur><div id=header-wrapper><nav><a href=/>devraza</a><button aria-label="toggle expand" class=separator id=toggler>::</button><span class="wrap left fold">{</span><a href=/blog>blog</a><span class="wrap-separator fold">,</span><a class=fold href=/projects>projects</a><span class="wrap right fold">} ;</span></nav><div id=btns><a aria-label="rss feed" href=/blog/feed.xml><svg viewbox="0 0 24 24" height=24 width=24 xmlns=http://www.w3.org/2000/svg><path d="M3 17C5.20914 17 7 18.7909 7 21H3V17ZM3 10C9.07513 10 14 14.9249 14 21H12C12 16.0294 7.97056 12 3 12V10ZM3 3C12.9411 3 21 11.0589 21 21H19C19 12.1634 11.8366 5 3 5V3Z" fill=currentColor></path></svg></a><button aria-label="theme switch" data-moon-icon='<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="24" height="24"><path d="M10 7C10 10.866 13.134 14 17 14C18.9584 14 20.729 13.1957 21.9995 11.8995C22 11.933 22 11.9665 22 12C22 17.5228 17.5228 22 12 22C6.47715 22 2 17.5228 2 12C2 6.47715 6.47715 2 12 2C12.0335 2 12.067 2 12.1005 2.00049C10.8043 3.27098 10 5.04157 10 7ZM4 12C4 16.4183 7.58172 20 12 20C15.0583 20 17.7158 18.2839 19.062 15.7621C18.3945 15.9187 17.7035 16 17 16C12.0294 16 8 11.9706 8 7C8 6.29648 8.08133 5.60547 8.2379 4.938C5.71611 6.28423 4 8.9417 4 12Z" fill="currentColor"></path></svg>' data-sun-icon='<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="24" height="24"><path d="M12 18C8.68629 18 6 15.3137 6 12C6 8.68629 8.68629 6 12 6C15.3137 6 18 8.68629 18 12C18 15.3137 15.3137 18 12 18ZM12 16C14.2091 16 16 14.2091 16 12C16 9.79086 14.2091 8 12 8C9.79086 8 8 9.79086 8 12C8 14.2091 9.79086 16 12 16ZM11 1H13V4H11V1ZM11 20H13V23H11V20ZM3.51472 4.92893L4.92893 3.51472L7.05025 5.63604L5.63604 7.05025L3.51472 4.92893ZM16.9497 18.364L18.364 16.9497L20.4853 19.0711L19.0711 20.4853L16.9497 18.364ZM19.0711 3.51472L20.4853 4.92893L18.364 7.05025L16.9497 5.63604L19.0711 3.51472ZM5.63604 16.9497L7.05025 18.364L4.92893 20.4
</span><span> </span><span style=color:#888># ...
</span><span> </span><span style=color:#ffb964>services</span><span>.</span><span style=color:#ffb964>searx </span><span>= {
</span><span> </span><span style=color:#ffb964>enable </span><span>= true;
</span><span> </span><span style=color:#ffb964>settings </span><span>= {
</span><span> </span><span style=color:#ffb964>server </span><span>= {
</span><span> </span><span style=color:#ffb964>port </span><span>= </span><span style=color:#cf6a4c>8888</span><span>;
</span><span> </span><span style=color:#ffb964>bind_address </span><span>= </span><span style=color:#99ad6a>"127.0.0.1"</span><span>;
</span><span> </span><span style=color:#ffb964>secret_key </span><span>= </span><span style=color:#99ad6a>"@SEARX_SECRET_KEY@"</span><span>;
</span><span> </span><span style=color:#ffb964>base_url </span><span>= </span><span style=color:#99ad6a>"https://search.devraza.duckdns.org/"</span><span>; </span><span style=color:#888># replace with wherever you want to host yours
</span><span> };
</span><span> };
</span><span> };
</span><span> </span><span style=color:#888># ...
</span><span>}
</span></code></pre><p>The snippet above starts the <code>searx</code> systemd service for listening on port <code>8888</code>, and assumes a <code>base_url</code> of <code>https://search.devraza.duckdns.org</code>.<p>Now that we've got the actual <code>searx</code> instance running, we can now set up a reverse proxy allowing the service to be accessed remotely (whether this is within your local network or across the internet is up to you).<h2 id=setting-up-a-reverse-proxy>Setting up a reverse proxy<a aria-label="Anchor link for: setting-up-a-reverse-proxy" class=zola-anchor href=#setting-up-a-reverse-proxy>#</a></h2><h3 id=what-is-a-reverse-proxy>What is a reverse proxy?<a aria-label="Anchor link for: what-is-a-reverse-proxy" class=zola-anchor href=#what-is-a-reverse-proxy>#</a></h3><p>Before I get started with the technical details of setting this up, I'd like to briefly clarify what a reverse proxy exactly is (to my understanding).<p>Let's get the wikipedia definition of reverse proxy out of the way first:<blockquote><p>[...] a reverse proxy is an application that sits in front of back-end applications and forwards client requests to those applications. [...]</blockquote><p>However, you might be confused as to what this actually means; I'll give an example of the usage of reverse proxies to better explain this:<ul><li>Suppose you've got a few services running on a server (for demonstration purposes, let's name these <code>x</code>, <code>y</code> and <code>z</code>), each running on their own unique port.<li>Assuming you had a domain, and wanted to access all of these services from their own unique sub-domains (e.g. <code>x.yourdomain.com</code>, <code>y.yourdomain.com</code> and <code>z.yourdomain.com</code>), you would have to use a reverse proxy.<li>This reverse proxy would take in requests from clients going to sub-domains, and forward these requests to the appropriate port on your machine for the service being requested.</ul><p>The concept should be clear now, if it wasn't already.<h3 id=using-nginx-to-set-up-the-reverse-proxy>Using NGINX to set up the reverse proxy<a aria-label="Anchor link for: using-nginx-to-set-up-the-reverse-proxy" class=zola-anchor href=#using-nginx-to-set-up-the-reverse-proxy>#</a></h3><p>NGINX is a popular web server that supports the creation of virtual hosts and the usage of reverse proxies. To accomodate our <code>searx</code> instance, we append the following to our NixOS server configuration:<pre class=language-nix data-lang=nix data-linenos style=background:#151515;color:#e8e8d3><code class=language-nix data-lang=nix><table><tbody><tr><td>1<td><span>{
</span><tr><td>2<td><span> </span><span style=color:#888># ...
</span><tr><td>3<td><span> </span><span style=color:#ffb964>services</span><span>.</span><span style=color:#ffb964>nginx </span><span>= {
</span><tr><td>4<td><span> </span><span style=color:#ffb964>enable </span><span>= true;
</span><tr><td>5<td><span> </span><span style=color:#888># any extra configuration here
</span><tr><td>6<td><span> </span><span style=color:#ffb964>virtualHosts </span><span>= {
</span><tr><td>7<td><span> </span><span style=color:#99ad6a>"search" </span><span>= { </span><span style=color:#888># this can be anything, being an arbitrary identifier
</span><tr><td>8<td><span> </span><span style=color:#ffb964>forceSSL </span><span>= true;
</span><tr><td>9<td><span> </span><span style=color:#ffb964>serverName </span><span>= </span><span style=color:#99ad6a>"search.yourdomain.com"</span><span>; </span><span style=color:#888># replace this with whatever you're serving from
</span><tr><td>10<td><span> </span><span style=color:#888># SearX proxy
</span><tr><td>11<td><span> </span><span style=color:#ffb964>locations</span><span>.</span><span style=color:#99ad6a>"/" </span><span>= {
</span><tr><td><mark style=background:#010101>12</mark><td><mark style=background:#010101><span> </span><span style=color:#ffb964>proxyPass </span><span>= </span><span style=color:#99ad6a>"http://${toString </span><span style=color:#ffb964>config</span><span style=color:#99ad6a>.</span><span style=color:#ffb964>services</span><span style=color:#99ad6a>.</span><span style=color:#ffb964>searx</span><span style=color:#99ad6a>.</span><span style=color:#ffb964>settings</span><span style=color:#99ad6a>.</span><span style=color:#ffb964>server</span><span style=color:#99ad6a>.</span><span style=color:#ffb964>bind_address</span><span style=color:#99ad6a>}:${toString </span><span style=color:#ffb964>config</span><span style=color:#99ad6a>.</span><span style=color:#ffb964>services</span><span style=color:#99ad6a>.</span><span style=color:#ffb964>searx</span><span style=color:#99ad6a>.</span><span style=color:#ffb964>settings</span><span style=color:#99ad6a>.</span><span style=color:#ffb964>server</span><span style=color:#99ad6a>.</span><span style=color:#ffb964>port</span><span style=color:#99ad6a>}"</span><span>;
</span></mark><tr><td>13<td><span> </span><span style=color:#ffb964>proxyWebsockets </span><span>= true;
</span><tr><td>14<td><span> </span><span style=color:#ffb964>recommendedProxySettings </span><span>= true;
</span><tr><td>15<td><span> };
</span><tr><td>16<td><span> };
</span><tr><td>17<td><span> };
</span><tr><td>18<td><span> };
</span><tr><td>19<td><span> </span><span style=color:#888># ...
</span><tr><td>20<td><span>}
</span></table></code></pre><blockquote class="callout note"><div class=icon><svg viewbox="0 0 24 24" height=20 width=20 xmlns=http://www.w3.org/2000/svg><path d="M12 22C6.47715 22 2 17.5228 2 12C2 6.47715 6.47715 2 12 2C17.5228 2 22 6.47715 22 12C22 17.5228 17.5228 22 12 22ZM12 20C16.4183 20 20 16.4183 20 12C20 7.58172 16.4183 4 12 4C7.58172 4 4 7.58172 4 12C4 16.4183 7.58172 20 12 20ZM11 7H13V9H11V7ZM11 11H13V17H11V11Z" fill=currentColor></path></svg></div><div class=content><p><strong>Note</strong><p>The expression highlighted above is used to dynamically adjust the location NGINX will forward requests to, depending on your <code>searx</code> config</div></blockquote><p>After saving your changes and rebuilding your server's system configuration (as usual), you should have a working <em>private</em> instance of SearXNG that you can access using the <code>serverName</code> you've given it.<p>Set your browser to use this as your search engine using the relevant documentation (with Firefox this is as easy as right-clicking on the URL after opening up the page and clicking a button). Enjoy!</article><div class=giscus></div></div><footer><div class=copyright><p>© 2024 Muhammad Nauman Raza</div><div class=credits>powered by <a rel="noreferrer noopener" href=https://www.getzola.org target=_blank>zola</a> and <a rel="noreferrer noopener" href=https://github.com/isunjn/serene target=_blank>serene</a></div></footer></main></div><script src=/js/lightense.min.js></script><script src=/js/main.js></script>