Nextcloud on S3 instead of a disk

A scan, 270 MB, a whole pile of paperwork squeezed into one PDF. I tap it in my Nextcloud, and instead of page one I get a message: “File too large”. Too large for what? And more to the point, is something wrong with the file?

Normally you’d check whether the file is still intact on the hard drive. Mine isn’t on any hard drive in my house. It sits in rented storage in a data centre, and over there it isn’t even called “scan”. It just has a number. How that works, why I built it this way and what it means for backups is what this piece is about, one step at a time and with no prior knowledge needed.

Your own Dropbox at home

Nextcloud is something like Dropbox or Google Drive, except you run it yourself. Your laptop and phone get a folder that syncs with a server, plus calendars and contacts. The difference is that the server stands in my home rather than at some big company.

More precisely, my Nextcloud runs in a VM, a virtual machine. That’s a computer my home server merely pretends to be, with its own operating system and its own hard drive, which is really just a file on the server. A good dozen of them run side by side on that one server, one per service. The one for Nextcloud is called nextcloud-01, and its hard drive is 40 GB.

40 GB is not much for photos, scans and everything else. Yet it has never filled up since Nextcloud moved in. That’s down to where the files actually go.

A storage unit at Hetzner

Hetzner is a German hosting company, a firm that rents out servers and storage in its data centres. One of its products is storage you talk to through an interface called S3. I’ll just call it S3 storage from here on. It works differently from a hard drive. There are no folders; you hand something over under a name and get it back under that name.

Think of it as a warehouse. I rent my own unit in it, which S3 calls a bucket. Into the unit go boxes, which are called objects. Each box has a label, and those labels are all the order there is. The warehouse doesn’t care what’s inside a box.

A file that is just a number

Nextcloud can use such a storage unit in two ways. Either as an extra folder next to your own files. Or as primary storage, meaning the place where all files end up instead of on the VM’s hard drive. I use the second.

Here is what then happens on every upload. Nextcloud writes the file’s name, folder, size and owner into its database, a kind of card index where every file gets a card with a running number. The file’s content goes into the bucket as an object, and the label says nothing but that number, for example urn:oid:4711. The picture follows one file from the top down:

So in the bucket, Taxes/2025.pdf turns into box 4711. Which box holds which file is written only on the cards in the database. That leaves the VM with just the database, the settings and the Nextcloud programs themselves, about 3.3 GB in all when I moved it in September. The bucket held 565 objects totalling a little over 8 GB at the time, and that part grows without the VM’s hard drive noticing.

Remember the card index; it’ll be back. Without it, the bucket is just a warehouse full of numbered boxes.

What I get, and what it costs

Sounds roundabout. Why not just a big hard drive? The trade-off fits in a small table, with what I gain on the left and what I pay on the right:

My main reason is the top left. All my VMs share my home server’s hard drive. If the Nextcloud files grew in there, sooner or later they’d eat the space of every other service. This way the bucket grows instead, and I never have to buy a drive for it. As a bonus, the files are already out of the house. If my home server dies, they don’t care.

Day to day, the price is a detour. Every file goes from the laptop to the VM and on to the data centre, and takes the same path back when I open it. So even when I’m on the sofa at home, each file leaves the house once. With big files, my internet connection’s upload speed sets the pace. I’ve honestly never measured how much waiting that adds up to.

The second price is a decision you only make once. Primary storage is chosen when you set Nextcloud up; you can’t simply switch a running one over. Changing to a different storage unit, on the other hand, is surprisingly easy, as the move showed.

The move in which no file moved

Until September 2026 my Nextcloud ran without a VM of its own, straight on my desktop computer. It ran as Nextcloud AIO, short for All-in-One, a complete package of containers. A container is a program that runs in its own box with everything it needs, walled off from the rest of the computer. AIO consists of ten such boxes: Nextcloud itself, the database, an office editor called EuroOffice and a few helpers. One of them, the master container, is the caretaker. It starts and updates the other nine by itself.

To do that, the master container needs full control over every container on the computer. I don’t want a caretaker like that next to other services, so for the move Nextcloud got a VM to itself, namely nextcloud-01.

On 8 September it was time. The data folders of the ten containers, the 3.3 GB mentioned above, travelled packed up over an encrypted connection to the new VM. A script set up the master container there, and after that Nextcloud reported itself as installed. And the files? They didn’t move. They were already in the bucket, and the new VM found them exactly where the old one had left them.

Its own storage unit in six seconds

Until then Nextcloud shared its bucket with other data from my homelab. That same evening it got one of its own. The copy ran server-side, meaning inside the data centre, without a single byte going through my connection: 565 objects, a little over 8 GB, six seconds.

Then came the actual switch, in four steps, all done with occ, Nextcloud’s command-line tool:

1. Put Nextcloud into maintenance mode, so it stops accepting changes.
2. Copy again, only what had arrived since the first copy.
3. Enter the new bucket’s name in the settings.
4. Switch maintenance mode off.

I left the old bucket and its objects in place as a way back. That kind of quick copy isn’t a given, by the way. A day later it failed between an older bucket and a new one, presumably because the two sit in different parts of the data centre. Then all the data has to pass through a computer that reads and rewrites it, which for those roughly 50 GB took two attempts.

Who gets in?

So the files sit at Hetzner and the card index knows their names. That leaves the question of who can get at them. For that I use single sign-on: one central login, like a key that fits several doors. In my case it’s a service called Authentik, running in its own VM. Nextcloud’s login page has a “Log in with Authentik” button, and only people I’ve put into Authentik’s Nextcloud group get through.

One setting looks like a footnote and is actually the most important. Nextcloud takes my username from Authentik exactly as it is. It has to, because in the card index every file belongs to a username. If Authentik turned up with a different one, Nextcloud would create a second, empty account, and my files would seem to be gone. Nothing would be deleted, but try explaining that to someone staring at an empty folder.

Once every login failed because Authentik wanted an extra setting after an update; part 2 has that story.

What the backups cover

Back to the question from the start, one size up. What if it isn’t one file that’s stuck, but the card index that’s broken? Then there are 565 boxes in the warehouse, and I have no idea which of them is my tax return. That makes the database the most valuable thing on the VM, and the backups are built around it.

I need two more terms for this. The first is the backup server, a second server, also at Hetzner, to which my home server backs up every VM as a whole each night. The second is versioning, a bucket setting. With it, an object that gets overwritten or deleted is kept as an older version, a bit like a recycle bin with a memory.

With those, the backups of nextcloud-01 fit into three rows:

The picture shows the same three paths, plus at the bottom the one that doesn’t exist yet:

For the 03:30 backup a script briefly stops every AIO container except the caretaker, so nothing changes while it packs. Afterwards the master container starts everything again. How that script handles all my other services is told in the deep dive on compose-backup.

There’s one thing I have to keep in mind when restoring. If I bring back last night’s card index, it doesn’t know today’s files. Their boxes then sit unlabelled in the warehouse. The other way round, it knows files I’ve deleted since, and that’s where versioning helps, because the deleted objects are still there as older versions.

One gap remains, and I don’t want to play it down. Versioning protects me from my own mistakes; it does nothing if the whole bucket or my Hetzner account is lost. The backup of the card index even sits in the same bucket as the files. So far there is no copy at a second provider. Off-site, it turns out, is not the same as backed up.

From practice: In mid-October the 03:30 backup failed every single night. The cause was a memory-saving feature. My home server can take working memory away from a VM when it needs it elsewhere, down to a minimum I set. For nextcloud-01 that minimum was 3 GB, and at times the VM was running on 2.8 GB. At 03:30 the copy program for the upload joined in with about 1 GB. For the next one to two hours the VM kept desperately rereading the same files from disk, until the system killed the copy program. I raised the minimum to 4 GB. The next night the upload went through, a little over 5 GB in just over an hour. Since then, whenever I set a minimum, I think about the night, not the day.

And the scan?

It was fine the whole time. Nextcloud hadn’t logged an error, and occ fetched the file from the bucket without complaint. Tapping it had opened the PDF in EuroOffice, the office editor from the AIO package, which rejects large files. Since PDFs now open in the built-in PDF viewer, the fat scan opens too.

What I took away is mostly an order of things. When something sticks in a Nextcloud on S3 storage, it’s tempting to blame the storage, simply because you can’t see it. First check whether the file can be read, then look for whoever refuses it. And I now treat the card index as what it is: the only thing that turns 565 numbers back into my files.

How backups work across the whole homelab, and who raises the alarm at night when a run fails, is in part 6: Backup and monitoring.