… So… Is it true or not?!

Well, I’ve solved it! I now have a web interface (accessible via VPN, although, in principle, I could expose it to the internet) that allows fast, full-text search of all my old emails. Here is the recipe:

1. Maildir: I converted all my mbox files to maildir using this python script: https://superuser.com/questions/1169371/how-to-convert-mbox-mail-files-as-found-in-thunderbird-dir-to-maildir#1343019
2. Installed notmuch via my distro’s repository and set it up (notmuch setup & notmuch new). This creates a new folder in your maildir directory containing full-text search info.
3. Installed netviel via python3 -m pip install netviel and then ran it via python3 -m netviel

That’s it! This let’s you search locally. I actually did a few more steps because I wanted to containerize this thing so I could run it on my NAS. I’d be happy to go into detail about that too, if you’re interested. One hiccup was that, for some reason, netviel binds to 127.0.0.1 instead of 0.0.0.0, and there is no way to change that without compiling the project yourself. But, I found a workaround for my Docker container where you can use socat bound to 0.0.0.0 to redirect requests to netviel, so that requests from other computers appear local to netviel.

Anyway, that makes it all sound more complicated than it is. I am super-pleased to have solved this problem at last!

Yes, I’m coming to similar conclusions myself. To be fair, encryption is a configurable option with Mailpiler. But, yes, it is all digested and stored in a mysql database, which is definitely more opaque than plaintext in the filesystem. I might try the mutt + notmuch solution described by @marty_relaxes@discuss.technics.de below. Sounds like it might be a challenge to set up but would work great forever after. I’ll need to figure out how to convert my mbox files to maildir, but Google suggests there are tools for that. Good luck to you, let us know what you ultimately figure out! I’ve been working on this off-and-on for a few months now without figuring our a good solution!

Edit: I guess, if you want fast full-text search, a database will have to enter the equation somewhere, though.

Alas, no! Things seemed to be going well: I got >90k messages imported from my Google Takeout mbox file before the import was interrupted (not mailpiler’s fault). At this point, I logged into the “auditor” account and was able to see my emails and search them. But, then I resumed the import. By the end of today, the import was finished (~150k messages total). When I logged in with the auditor account, I got some error “No search results” and nothing I could do about it. This is actually what happened last time I tried mailpiler, too, now that I recall. All seemed fine, but, it seems, the database got corrupted or something along the way… So, now it’s useless. I might try it one more time over the next few days. I’ll keep y’all posted.

I am currently working on this. Finally got the Docker working and am importing my 15GB mbox as we speak! I’ll post back here about how it works out.

I have solved this problem! The trick is to use two Docker containers:

1. Gluetun (https://github.com/qdm12/gluetun): set this up to connect to your VPN.
2. Tailscale (https://tailscale.com/kb/1282/docker/): set this to use the Gluetun network.

Here is an example docker-compose.yml:

version: "3"
services:
  gluetun:
    image: qmcgaw/gluetun
    container_name: gluetun
    # line above must be uncommented to allow external containers to connect.
    # See https://github.com/qdm12/gluetun-wiki/blob/main/setup/connect-a-container-to-gluetun.md#external-container-to-gluetun
    restart: unless-stopped
    cap_add:
      - NET_ADMIN
    devices:
      - /dev/net/tun:/dev/net/tun
    volumes:
      - ./gluetun:/gluetun
    environment:
      - VPN_SERVICE_PROVIDER=airvpn
      - VPN_TYPE=wireguard
      - WIREGUARD_PRIVATE_KEY=xxx
      - WIREGUARD_PRESHARED_KEY=xxx
      - WIREGUARD_ADDRESSES=xxx
      - WIREGUARD_MTU=1320
      - SERVER_COUNTRIES=United States
      # See https://github.com/qdm12/gluetun-wiki/tree/main/setup#setup
      # Timezone for accurate log times
      - TZ=America/New_York
      # Server list updater
      # See https://github.com/qdm12/gluetun-wiki/blob/main/setup/servers.md#update-the-vpn-servers-list
      - UPDATER_PERIOD=24h

  tailscale:
    container_name: tailscale
    cap_add:
      - NET_ADMIN
      - NET_RAW
    volumes:
      - ./tailscale/var/lib:/var/lib
      - ./tailscale/state:/state
      - /dev/net/tun:/dev/net/tun
    network_mode: "service:gluetun"
    restart: unless-stopped
    environment:
      - TS_HOSTNAME=airvpn-exit-node
      - TS_AUTHKEY=xxxxxxxx
      - TS_EXTRA_ARGS=--login-server=https://example.com --advertise-exit-node
      - TS_NO_LOGS_NO_SUPPORT=true
      - TS_STATE_DIR=/state
    image: tailscale/tailscale

It’s too bad Android didn’t lean more into it’s relative freedom. Instead, almost all Android manufacturers have followed Apple’s lead closely. I have to believe there’d be a sizeable market for a flagship Android phone with, say, a removable battery, headphone jack, SD card slot, and an easily unlocked bootloader.