devraza/website
1
Fork 0
Code Issues Pull requests Packages Projects Releases Wiki Activity
website/public/blog/selfhost-search-engine/index.html

36 lines
17 KiB
HTML
Raw Permalink Blame History

<!doctype html><html lang=en><head><meta charset=utf-8><meta content="width=device-width,initial-scale=1.0" name=viewport><meta content="light dark" name=color-scheme><title>Host your own private search engine with SearXNG</title><link href=/img/favicon-32x32.png rel=icon sizes=32x32 type=image/png><link href=/img/favicon-16x16.png rel=icon sizes=16x16 type=image/png><link href=/img/apple-touch-icon.png rel=apple-touch-icon sizes=180x180><link href=https://fonts.googleapis.com rel=preconnect><link crossorigin href=https://fonts.gstatic.com rel=preconnect><link href="https://fonts.googleapis.com/css2?family=Signika&display=swap" rel=stylesheet><style>*{font-family:monospace!important}body{--primary-color:#8070c6;--primary-pale-color:#8070c61c;--text-color:#151517;--text-pale-color:#454449;--bg-color:#f4f0f3;--highlight-mark-color:#5f75b045;--callout-note-color:#e887bb;--callout-important-color:#a292e8;--callout-warning-color:#d9d564;--callout-alert-color:#f06969;--callout-question-color:#78b9c4;--callout-tip-color:#91d65c}body.dark{--primary-color:#a292e8;--primary-pale-color:#a292e81c;--text-color:#ece5ea;--text-pale-color:#5c5c61;--bg-color:#151517;--highlight-mark-color:#5f75b045;--callout-note-color:#e887bb;--callout-important-color:#a292e8;--callout-warning-color:#d9d564;--callout-alert-color:#f06969;--callout-question-color:#78b9c4;--callout-tip-color:#91d65c}body{--main-font:'Signika',ui-sans-serif,system-ui,-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,"Helvetica Neue",Arial,"Noto Sans",sans-serif,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol","Noto Color Emoji";--code-font:ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,"Liberation Mono","Courier New",monospace;--homepage-max-width:750px;--main-max-width:750px;--avatar-size:70px;--avatar-radius:0;--paragraph-font-size:18px;--paragraph-line-height:1.75;--aside-font-size:16px;--img-border-radius:0;--inline-code-border-radius:2px}</style><link href=/main.css rel=stylesheet><body class=post><script>if(localStorage.getItem('theme')=='dark'){document.body.classList.add('dark');const a=document.querySelector('link#hl');if(a)a.href='/hl-dark.css'}</script><header class=blur><div id=header-wrapper><nav><a href=/>devraza</a><button aria-label="toggle expand" class=separator id=toggler>::</button><span class="wrap left fold">{</span><a href=/blog>blog</a><span class="wrap-separator fold">,</span><a class=fold href=/projects>projects</a><span class="wrap right fold">} ;</span></nav><div id=btns><a aria-label="rss feed" href=/blog/feed.xml><svg viewbox="0 0 24 24" height=24 width=24 xmlns=http://www.w3.org/2000/svg><path d="M3 17C5.20914 17 7 18.7909 7 21H3V17ZM3 10C9.07513 10 14 14.9249 14 21H12C12 16.0294 7.97056 12 3 12V10ZM3 3C12.9411 3 21 11.0589 21 21H19C19 12.1634 11.8366 5 3 5V3Z" fill=currentColor></path></svg></a><button aria-label="theme switch" data-moon-icon='<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="24" height="24"><path d="M10 7C10 10.866 13.134 14 17 14C18.9584 14 20.729 13.1957 21.9995 11.8995C22 11.933 22 11.9665 22 12C22 17.5228 17.5228 22 12 22C6.47715 22 2 17.5228 2 12C2 6.47715 6.47715 2 12 2C12.0335 2 12.067 2 12.1005 2.00049C10.8043 3.27098 10 5.04157 10 7ZM4 12C4 16.4183 7.58172 20 12 20C15.0583 20 17.7158 18.2839 19.062 15.7621C18.3945 15.9187 17.7035 16 17 16C12.0294 16 8 11.9706 8 7C8 6.29648 8.08133 5.60547 8.2379 4.938C5.71611 6.28423 4 8.9417 4 12Z" fill="currentColor"></path></svg>' data-sun-icon='<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="24" height="24"><path d="M12 18C8.68629 18 6 15.3137 6 12C6 8.68629 8.68629 6 12 6C15.3137 6 18 8.68629 18 12C18 15.3137 15.3137 18 12 18ZM12 16C14.2091 16 16 14.2091 16 12C16 9.79086 14.2091 8 12 8C9.79086 8 8 9.79086 8 12C8 14.2091 9.79086 16 12 16ZM11 1H13V4H11V1ZM11 20H13V23H11V20ZM3.51472 4.92893L4.92893 3.51472L7.05025 5.63604L5.63604 7.05025L3.51472 4.92893ZM16.9497 18.364L18.364 16.9497L20.4853 19.0711L19.0711 20.4853L16.9497 18.364ZM19.0711 3.51472L20.4853 4.92893L18.364 7.05025L16.9497 5.63604L19.0711 3.51472ZM5.63604 16.9497L7.05025 18.364L4.92893 20.4853L3.51472 19.0711L5.63604 16.9497ZM23 11V13H20V11H23ZM4 11V13H1V11H4Z" fill="currentColor"></path></svg>' id=theme-toggle><svg viewbox="0 0 24 24" height=24 width=24 xmlns=http://www.w3.org/2000/svg><path d="M10 7C10 10.866 13.134 14 17 14C18.9584 14 20.729 13.1957 21.9995 11.8995C22 11.933 22 11.9665 22 12C22 17.5228 17.5228 22 12 22C6.47715 22 2 17.5228 2 12C2 6.47715 6.47715 2 12 2C12.0335 2 12.067 2 12.1005 2.00049C10.8043 3.27098 10 5.04157 10 7ZM4 12C4 16.4183 7.58172 20 12 20C15.0583 20 17.7158 18.2839 19.062 15.7621C18.3945 15.9187 17.7035 16 17 16C12.0294 16 8 11.9706 8 7C8 6.29648 8.08133 5.60547 8.2379 4.938C5.71611 6.28423 4 8.9417 4 12Z" fill=currentColor></path></svg></button><button aria-label="table of content" id=toc-toggle><svg viewbox="0 0 24 24" height=24 width=24 xmlns=http://www.w3.org/2000/svg><path d="M3 4H21V6H3V4ZM3 11H15V13H3V11ZM3 18H21V20H3V18Z" fill=currentColor></path></svg></button></div></div></header><div id=wrapper><div id=blank></div><aside class=blur><nav><ul><li><a class=h2 href=#introduction>Introduction</a><li><a class=h2 href=#setting-up-searxng>Setting up SearXNG</a> <ul><li><a class=h3 href=#installing-the-service>Installing the service</a><li><a class=h3 href=#setting-up-a-reverse-proxy>Setting up a reverse proxy</a></ul></ul></nav><button aria-label="back to top" id=back-to-top><svg viewbox="0 0 24 24" height=24 width=24 xmlns=http://www.w3.org/2000/svg><path d="M11.9997 10.8284L7.04996 15.7782L5.63574 14.364L11.9997 8L18.3637 14.364L16.9495 15.7782L11.9997 10.8284Z" fill=currentColor></path></svg></button></aside><main><div><div data-check-icon='<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="24" height="24"><path d="M10.0007 15.1709L19.1931 5.97852L20.6073 7.39273L10.0007 17.9993L3.63672 11.6354L5.05093 10.2212L10.0007 15.1709Z" fill="currentColor"></path></svg>' data-copy-icon='<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="24" height="24"><path d="M6.9998 6V3C6.9998 2.44772 7.44752 2 7.9998 2H19.9998C20.5521 2 20.9998 2.44772 20.9998 3V17C20.9998 17.5523 20.5521 18 19.9998 18H16.9998V20.9991C16.9998 21.5519 16.5499 22 15.993 22H4.00666C3.45059 22 3 21.5554 3 20.9991L3.0026 7.00087C3.0027 6.44811 3.45264 6 4.00942 6H6.9998ZM5.00242 8L5.00019 20H14.9998V8H5.00242ZM8.9998 6H16.9998V16H18.9998V4H8.9998V6Z" fill="currentColor"></path></svg>' id=copy-cfg style=display:none></div><article data-backlink-icon='<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" width="20" height="20"><path d="M9.41421 8L18.0208 16.6066L16.6066 18.0208L8 9.41421V17H6V6H17V8H9.41421Z" fill="currentColor"></path></svg>' class=prose><h1>Host your own private search engine with SearXNG</h1><div id=post-info><div id=date><span id=publish>2023-12-31</span></div><div id=tags><a href=https://devraza.giize.com/tags/searxng><span>#</span>searxng</a><a href=https://devraza.giize.com/tags/privacy><span>#</span>privacy</a><a href=https://devraza.giize.com/tags/selfhosted><span>#</span>selfhosted</a><a href=https://devraza.giize.com/tags/nixos><span>#</span>nixos</a></div></div><h1 id=introduction>Introduction<a aria-label="Anchor link for: introduction" class=zola-anchor href=#introduction>#</a></h1><p><a rel="nofollow noreferrer" href=https://docs.searxng.org/>SearXNG</a>, put in its own words, is a 'free internet metasearch engine'. Note that it describes itself as a <em>metasearch</em> engine specifically - unlike your traditional search engine like Google or Bing, SearXNG does things a little bit differently: It aggregrates the results produced by search services like those aforementioned, and feeds them back to you.<p>Because of this key detail and a great deal of effort by those who've helped shape it, SearXNG protects your privacy, and does so very well:<ul><li>Private data from requests going to the search services it aggregrates results from is removed<li>It does <strong>not</strong> forward <em>anything</em> to any third parties through search services<li>Private data is <em>also</em> removed from requests going to the results pages</ul><p>Furthermore, SearXNG can be configured to use <a rel="nofollow noreferrer" href=https://torproject.org>Tor</a>.<p>However, the aspect of privacy isn't the only great selling feature of the engine; from my use of the engine so far, it's also great at...searching (is that a surprise?). The fact that it's a metasearch engine plays a key role in this, as it provides SearXNG the ability to pull content more efficiently and gives <em>you</em> the ability to further tailor your experience.<h1 id=setting-up-searxng>Setting up SearXNG<a aria-label="Anchor link for: setting-up-searxng" class=zola-anchor href=#setting-up-searxng>#</a></h1><h2 id=installing-the-service>Installing the service<a aria-label="Anchor link for: installing-the-service" class=zola-anchor href=#installing-the-service>#</a></h2><p>As you may have expected if you've used NixOS for a while, searxng is packaged <em>and</em> has a service on NixOS. This makes setting it up just that much easier.<p>To get started, place somewhere in your <em>system</em> config the following:<pre class=language-nix data-lang=nix style=background:#151515;color:#e8e8d3><code class=language-nix data-lang=nix><span>{
</span><span> </span><span style=color:#888># ...
</span><span> </span><span style=color:#ffb964>services</span><span>.</span><span style=color:#ffb964>searx </span><span>= {
</span><span> </span><span style=color:#ffb964>enable </span><span>= true;
</span><span> </span><span style=color:#ffb964>settings </span><span>= {
</span><span> </span><span style=color:#ffb964>server </span><span>= {
</span><span> </span><span style=color:#ffb964>port </span><span>= </span><span style=color:#cf6a4c>8888</span><span>;
</span><span> </span><span style=color:#ffb964>bind_address </span><span>= </span><span style=color:#99ad6a>"127.0.0.1"</span><span>;
</span><span> </span><span style=color:#ffb964>secret_key </span><span>= </span><span style=color:#99ad6a>"@SEARX_SECRET_KEY@"</span><span>;
</span><span> </span><span style=color:#ffb964>base_url </span><span>= </span><span style=color:#99ad6a>"https://search.devraza.duckdns.org/"</span><span>; </span><span style=color:#888># replace with wherever you want to host yours
</span><span> };
</span><span> };
</span><span> };
</span><span> </span><span style=color:#888># ...
</span><span>}
</span></code></pre><p>The snippet above starts the <code>searx</code> systemd service for listening on port <code>8888</code>, and assumes a <code>base_url</code> of <code>https://search.devraza.duckdns.org</code>.<p>Now that we've got the actual <code>searx</code> instance running, we can now set up a reverse proxy allowing the service to be accessed remotely (whether this is within your local network or across the internet is up to you).<h2 id=setting-up-a-reverse-proxy>Setting up a reverse proxy<a aria-label="Anchor link for: setting-up-a-reverse-proxy" class=zola-anchor href=#setting-up-a-reverse-proxy>#</a></h2><h3 id=what-is-a-reverse-proxy>What is a reverse proxy?<a aria-label="Anchor link for: what-is-a-reverse-proxy" class=zola-anchor href=#what-is-a-reverse-proxy>#</a></h3><p>Before I get started with the technical details of setting this up, I'd like to briefly clarify what a reverse proxy exactly is (to my understanding).<p>Let's get the wikipedia definition of reverse proxy out of the way first:<blockquote><p>[...] a reverse proxy is an application that sits in front of back-end applications and forwards client requests to those applications. [...]</blockquote><p>However, you might be confused as to what this actually means; I'll give an example of the usage of reverse proxies to better explain this:<ul><li>Suppose you've got a few services running on a server (for demonstration purposes, let's name these <code>x</code>, <code>y</code> and <code>z</code>), each running on their own unique port.<li>Assuming you had a domain, and wanted to access all of these services from their own unique sub-domains (e.g. <code>x.yourdomain.com</code>, <code>y.yourdomain.com</code> and <code>z.yourdomain.com</code>), you would have to use a reverse proxy.<li>This reverse proxy would take in requests from clients going to sub-domains, and forward these requests to the appropriate port on your machine for the service being requested.</ul><p>The concept should be clear now, if it wasn't already.<h3 id=using-nginx-to-set-up-the-reverse-proxy>Using NGINX to set up the reverse proxy<a aria-label="Anchor link for: using-nginx-to-set-up-the-reverse-proxy" class=zola-anchor href=#using-nginx-to-set-up-the-reverse-proxy>#</a></h3><p>NGINX is a popular web server that supports the creation of virtual hosts and the usage of reverse proxies. To accomodate our <code>searx</code> instance, we append the following to our NixOS server configuration:<pre class=language-nix data-lang=nix data-linenos style=background:#151515;color:#e8e8d3><code class=language-nix data-lang=nix><table><tbody><tr><td>1<td><span>{
</span><tr><td>2<td><span> </span><span style=color:#888># ...
</span><tr><td>3<td><span> </span><span style=color:#ffb964>services</span><span>.</span><span style=color:#ffb964>nginx </span><span>= {
</span><tr><td>4<td><span> </span><span style=color:#ffb964>enable </span><span>= true;
</span><tr><td>5<td><span> </span><span style=color:#888># any extra configuration here
</span><tr><td>6<td><span> </span><span style=color:#ffb964>virtualHosts </span><span>= {
</span><tr><td>7<td><span> </span><span style=color:#99ad6a>"search" </span><span>= { </span><span style=color:#888># this can be anything, being an arbitrary identifier
</span><tr><td>8<td><span> </span><span style=color:#ffb964>forceSSL </span><span>= true;
</span><tr><td>9<td><span> </span><span style=color:#ffb964>serverName </span><span>= </span><span style=color:#99ad6a>"search.yourdomain.com"</span><span>; </span><span style=color:#888># replace this with whatever you're serving from
</span><tr><td>10<td><span> </span><span style=color:#888># SearX proxy
</span><tr><td>11<td><span> </span><span style=color:#ffb964>locations</span><span>.</span><span style=color:#99ad6a>"/" </span><span>= {
</span><tr><td><mark style=background:#010101>12</mark><td><mark style=background:#010101><span> </span><span style=color:#ffb964>proxyPass </span><span>= </span><span style=color:#99ad6a>"http://${toString </span><span style=color:#ffb964>config</span><span style=color:#99ad6a>.</span><span style=color:#ffb964>services</span><span style=color:#99ad6a>.</span><span style=color:#ffb964>searx</span><span style=color:#99ad6a>.</span><span style=color:#ffb964>settings</span><span style=color:#99ad6a>.</span><span style=color:#ffb964>server</span><span style=color:#99ad6a>.</span><span style=color:#ffb964>bind_address</span><span style=color:#99ad6a>}:${toString </span><span style=color:#ffb964>config</span><span style=color:#99ad6a>.</span><span style=color:#ffb964>services</span><span style=color:#99ad6a>.</span><span style=color:#ffb964>searx</span><span style=color:#99ad6a>.</span><span style=color:#ffb964>settings</span><span style=color:#99ad6a>.</span><span style=color:#ffb964>server</span><span style=color:#99ad6a>.</span><span style=color:#ffb964>port</span><span style=color:#99ad6a>}"</span><span>;
</span></mark><tr><td>13<td><span> </span><span style=color:#ffb964>proxyWebsockets </span><span>= true;
</span><tr><td>14<td><span> </span><span style=color:#ffb964>recommendedProxySettings </span><span>= true;
</span><tr><td>15<td><span> };
</span><tr><td>16<td><span> };
</span><tr><td>17<td><span> };
</span><tr><td>18<td><span> };
</span><tr><td>19<td><span> </span><span style=color:#888># ...
</span><tr><td>20<td><span>}
</span></table></code></pre><blockquote class="callout note"><div class=icon><svg viewbox="0 0 24 24" height=20 width=20 xmlns=http://www.w3.org/2000/svg><path d="M12 22C6.47715 22 2 17.5228 2 12C2 6.47715 6.47715 2 12 2C17.5228 2 22 6.47715 22 12C22 17.5228 17.5228 22 12 22ZM12 20C16.4183 20 20 16.4183 20 12C20 7.58172 16.4183 4 12 4C7.58172 4 4 7.58172 4 12C4 16.4183 7.58172 20 12 20ZM11 7H13V9H11V7ZM11 11H13V17H11V11Z" fill=currentColor></path></svg></div><div class=content><p><strong>Note</strong><p>The expression highlighted above is used to dynamically adjust the location NGINX will forward requests to, depending on your <code>searx</code> config</div></blockquote><p>After saving your changes and rebuilding your server's system configuration (as usual), you should have a working <em>private</em> instance of SearXNG that you can access using the <code>serverName</code> you've given it.<p>Set your browser to use this as your search engine using the relevant documentation (with Firefox this is as easy as right-clicking on the URL after opening up the page and clicking a button). Enjoy!</article><div class=giscus></div></div><footer><div class=copyright><p>© 2023-2024 Muhammad Nauman Raza</div><div class=credits>powered by <a rel="noreferrer noopener" href=https://www.getzola.org target=_blank>zola</a> and <a rel="noreferrer noopener" href=https://github.com/isunjn/serene target=_blank>serene</a></div></footer></main></div><script src=/js/lightense.min.js></script><script src=/js/main.js></script>
Reference in a new issue View git blame Copy permalink