Friendica Archive Building and Browsing

Due to various circumstances I got sidetracked from getting the Kyanite Facebook Archive Browser out the door this month as intended. Part of that was because I found out that my Friendica instance was shutting down at the end of the month. Friendica is one of many projects on “The Fediverse” alternatives to the big-tech owned platforms (Facebook, Twitter, Instagram, etc.). While it has many advantages one of the disadvantages is that most of the servers that you would join through are run by one or two people. That means when they decide to pull up stakes you have to figure out what to do with your fediverse digital existence too. That’s where I found myself at the beginning of the month. My answer was to take my new found skills with Dart and Flutter and the Kyanite code base to create an archival and archive viewing system for Friendica. The archiver and browser can be found at this link

Generating an Archive

First lets talk about generating the archive. The archive generator is a command line tool that uses the Friendica API to query for your timeline. You provide your username and password credentials (sorry OpenID logins don’t work right now), the name of your server, and the directory the archive will be put into. It then goes about archiving your entire timeline from the most recent to very first post and comment. So it will be something like:

./friendica_archiver --username user1 --server-name myfriendicaserver.com --archive-folder ~/Downloads/FriendicaArchive

Along with the posts and comments it will also by default download any attached images for the post as well. That way you have a complete archive of all of your personal content from the server. The default also executes until it runs to the end of the timeline or until it hits 20 billion post/comments. The user has flags they can use to change that behavior, including being able to restart it mid-way through.

For an active account I recommend running it with the standard defaults and watching for when you start seeing duplicate posts coming up in the prompts. Because Friendica allows changing the most recent ones may not be literal duplicates but updates. However if you know you didn’t edit posts or comments back far enough you know you are just doing unnecessary duplicate queries. In the future the logic may get more complicated to automatically stop when it “hit the already archived zone”.

When this is completed you will now have a folder with a structure of:

├── images
├── images.json
└── postsAndComments.json

The images folder contains all of the archived images. The images.json file maps the image file names, which are UUIDs, to URLs that they came from. The postsAndComments.json is an archive of the raw JSON responses from the server. This top level folder is what you will point your archive viewer to for browsing.

Browsing your Archive

With your archive ready to go you can download the Browser. On initial startup it asks you to choose the folder to load, which it will remember on startup the next time. You can jump to your posts view:

This shows your post text, actually the Friendica-generated HTML data which will include previews, some of the tweet structure etc. For posts or comments with photos there is also a photo gallery option so you can cycle through them. In most cases the HTML preview has the images as well which creates a duplicate image view unfortunately. It will also include how many likes/dislikes it has. If you hover over the like or dislike icon you can see who liked/disliked it as well. It will also show any comments you made on that post as well, although it will not show comments by other users. Lastly there is a copy button for copying the text and data to the clipboard and a link button to launch a browser with the original post or comment location. Like with Kyanite there is a full search capability to allow you to easily find content. Below is a screenshot showing such a search and what embedded comments look like:

For comments that are on posts other than your own or on comments on your post there is an “Orphaned Comments” view. All of the controls are the same as with posts and comments on the comments page, including the ability to launch the browser to the original comment on your server:

One thing I like to do is look at my posting habits over time. Since there is also data about how people interact with your posts I also added the ability to see who the top interacting accounts are. Below are two screenshots showing the gamut of statistics available to you. In the case of the “top interactors” clicking on the name launches their profile page:

Conclusion

With these two tools it is possible to archive your post and comment data from your Friendica instance for your personal use. With the browser it is also possible to have searching capabilities which are far better than what one is capable of doing with server side search and browsing.