Expiring old entries - ever growing database problem
Roland Haeder
Public
Seen by 71
The database is constantly growing, even with one active user, not sharing with and being shared with much.
Since I have started running my pod, according to phpMyAdmin the whole database has grown to ~2,661,012 entries with a data usage of 2.8 GiB (!) and only my "little" pod.
The feature should be configurable through yaml config or admin interface.
Updated from comment poster:
-->Only the pod of the original poster is obliged to store the post forever. The other pods can expire the post if no user fetched it for a certain time period. If the pod expires the post it removes everything except the posts metadata from the database. The metadata includes all the stuff needed to keep the post searchable (id, posting user, hashtags). If the pod ever needs the post again it will fetch it from the original posters pod.
This should significantly reduce the database size (especially for small pods) while keeping all posts accessible if needed.
Roland Haeder Sat 12 Mar 2016 5:21PM
I like this idea, proposal is edited with your idea.
CSammy Tue 15 Mar 2016 11:35AM
You may be interested in this: https://github.com/diaspora/diaspora/issues/4920
In short: The problem is known and being worked on. It actually seems not to be caused by the number of posts but by other meta data.
Also, although currently postponed, a re-fetch feature has been talked about. It may get implemented in the future.
Deleted User Tue 31 May 2016 10:30AM
Maybe there could be a way to set an expiry on a post, for example if I post about an event next week, there could be a way to set event expires and as poster it can be deleted.
Flaburgan Wed 1 Jun 2016 9:58AM
For your information, #4920 (linked above by @CSammy) is now closed in the develop branch. The next diaspora* major release (0.6.0.0) will drastically reduce the disk space used by the database.
Manuel Vögele · Sat 12 Mar 2016 8:49AM
I could think of the following system: Only the pod of the original poster is obliged to store the post forever. The other pods can expire the post if no user fetched it for a certain time period. If the pod expires the post it removes everything except the posts metadata from the database. The metadata includes all the stuff needed to keep the post searchable (id, posting user, hashtags). If the pod ever needs the post again it will fetch it from the original posters pod.
This should significantly reduce the database size (especially for small pods) while keeping all posts accessible if needed.