Robsie's Journal

You can send small files, such as photos and documents, conveniently as email attachments, as long as they are not too large or too many. If you send your 120 MB PhD thesis as a Word file to your professor, she might hate you; if you are lucky, the email server rejects your bloat mail before, and you just are wondering the next weeks why you don't get a response. A compressed PDF below 2 Mb might have more success. The same is true for emails with plenty of files as attachments: they can be annoying and quickly eat up precious email inbox space and network resources (well, the same is true for HTML emails, with signature images, trackers, and other 'features,' but that is a different topic).

Just remember: Email was not designed as a file-sharing program!

Several commercial platforms, such as Box, Dropbox, and Google Drive, offer file-sharing and hosting solutions. Usually, there is a free tier with limited cloud space and paid plans with more space and additional features.

NextCloud

NextCloud is a free and open source (FOSS) that you can self-host on your server. The space scales with your hardware, and you can install many additional apps, such as groupwork programs, editors, a bookmark manager, etc. In addition, you can synchronize your contacts and calendars (CardDAV and CalDAV). I've been using NextCloud for over a decade, and it has never disappointed me. I pay 5 EUR for 500 GB of server space, which is enough for my personal and professional needs. For collecting homework, I can open a directory for upload; for sending large datasets, I can share files or directories for download only; and we can also edit shared manuscripts in collaboration. I could praise NextCloud even more, but this post is about file sharing.

The following screenshot shows the dialog for sharing ~100 coffee genomes, totaling 25.4 GB.

In Thunderbird, the plugin 'cloud' enables the upload of large attachments to your NextCloud instance. Instead of the file, only the link is sent. The person receiving your mail will appreciate the small size of your message and can decide whether to download the file. With this approach, you keep large attachments separated from the email servers (saving bandwidth and disc space and improving privacy) and keep a backup on your private NextCloud. The configuration is straightforward:

Old school tools

I omit the classic methods, such as defining a download directory on your server and using command line interface (CLI) tools such as rsync (one of my favorites). They are well-documented and efficient but more suitable for nerds and server administrators who want to automate file transfers (e.g., for backups) and dislike graphical user interfaces (GUIs).

Now, I will present two peer-to-peer (P2P) solutions that don't require a server as a middleman:

Torrents

Torrent distribution efficiently shares large files by breaking them into pieces for simultaneous download from multiple peers. Popular clients include Transmission and qBittorrent, with web plugins like WebTorrent also available. Users can download via .torrent files or magnetic links containing all necessary metadata. Trackers coordinate peer communication while creating a torrent, which involves using a client's tools to specify files, tracker info, and settings. This generates a .torrent file for distribution. Torrents are useful for distributing large files such as ISO images for class exercises or non-copyrighted movies. While efficient and scalable, torrent technology raises privacy and security concerns, as the user's IP addresses are visible to peers and the files are not encrypted by default. There are also risks of malware and copyright issues. Heavy usage can lead to a high network load.

For example, this method can efficiently distribute the Linux ISO for mass spectrometry data processing MASSyPupX. The download can be initiated by simply opening the magnetic link magnetic link with a torrent client.

The screenshot shows the distribution of the ISO using the torrent: the computer with the web interface is the seeder (providing the file), and the BitTorrent client is the leecher (receiving data)

Torrent is an excellent choice for distributing large files to the public or multiple users. Besides finding torrent links on webpages or getting them passed through personal communication, you can search for torrents on search engines like LibreY.

SyncThing

However, if you want to exchange data only between a couple of devices and have higher requirements for security and privacy, SyncThing is your friend. Syncthing is an open-source, peer-to-peer file synchronization tool that allows secure file sharing across multiple devices without a central server. Unlike torrents, it offers enhanced privacy and control over shared data. Syncthing can be faster for small files or local network transfers, while torrents may be advantageous for large-scale distribution. With its end-to-end encryption and device verification, Syncthing provides superior security compared to most torrent protocols. It is an excellent choice for users prioritizing data privacy and seamless file synchronization across personal and trustful devices. The web interface allows for the registration of additional devices and the management of synchronized folders. The following screenshot shows the web interface for a computer with four peers:

SyncThing works well for transferring and synchronizing large (several GB) private files, such as patient data. But you can also set up a SynthThing-Fork folder on your Android phone and quickly upload files for mobile use, such as bus tickets or your encrypted KeePass password database.

Of course, these P2P transfers will fail if no files providing peer is online. You can mitigate this risk by synchronizing to an always-on server. However, you need additional resources and more precautions to secure your privacy for such a setup.

Good luck with your private file transfers!

How many web searches do you perform every day? 10, 50, 100? Which search engine are you using? Are you logged in, for example, to your Google account, when you are searching? Is your browser preventing your identification by fingerprinting, and are you using a VPN to hide your location?

You should know that commercial web search providers probably get more information from your search input than you get back as results. Additionally, if you use a search as an identified user, the results might already be adjusted to your shopping interests, political views, etc. At the end of the day, they live from selling your data (Surprise!). What can you do?

Of course, you should use a privacy-oriented browser and use a VPN. I personally mainly use the browsers Firefox and Brave for different purposes, and the Surfshark VPN.

However, the weak point is often the search engine. If you're still using Google search, even worse logged in with your profile, nobody can help you. At the very least, you should switch your default search engine to, for example, DuckDuckGo. It helps hide your identity and prevent tracking. Furthermore, you can search with a scope, e.g., “!gsc famous professor” will search for the famous professor's content in Google Scholar.

If you're even more paranoid, you can use a self-hosted web search engine. For my own and community use, I'm hosting two web search engines:

1. The first, https://libre-find.online/, is a LibreY instance, which gives you results from Brave Search, DuckDuckGo, Ecosia, Google, Mojeek, and Yandex Search. Additionally, you can search for images, videos, torrents, and maps. You can also connect to Libre-Find.Online with a Tor-compatible browser such as the Tor Browser or Brave, using the address http://ug3lz3wdjxljv5fxyoicgcugmcsuym5e4zgxggotjdscdrdwid7s2mqd.onion/.
2. The second search engine I'm self-hosting is a YaCy instance at https://yacy.nube-gran.de. YaCy is not only a peer-to-peer (P2P) search but also a web crawler, i.e., the instance can index pages that might be less interesting to commercial crawlers and web searches. Since YaCy instances connect to each other, a distributed search index is available for P2P searches. You can see the integration of my instance in the YaCy network at https://yacy.nube-gran.de/Network.html. The name of my node is 'PRESIDENTA' in honor of the first female president of Mexico, Dr. Claudia Sheinbaum Pardo.

Good luck with your private web searches!