Support user-specified expiration times #72

Ember · 2022-11-22T22:47:23+01:00

Ember commented

2022-11-22 22:47:23 +01:00

Closes #33

This pull request adds support for an additional field during file uploads: expires=<time>. This field can be set to either:

A time in hours for which the file should be retained
A timestamp in epoch milliseconds at which the file should be expired

Neither option allows setting an expiration time longer than what would naturally be allotted to the file. In other words, this allows users to shorten the retention time of their files, but not extend it.

In order to accomadate this change, the following changes were necessary:

Adding an expiration field to the file table in the database. Prior to this change, expiration dates were tracked using filesystem metadata. Unfortunately, I do not know a way to use filesystem metadata to store another date/duration to expire the file at, so it was necessary to replace this method of tracking expirations with the database
The creation of a migration script to facilitate the above database change.
As a result of the above two changes, the cleanup.py script needed to be overhauled. Because it now needs database connectivity, I elected to move it into the main flask app as an alternate entrypoint, which can now be executed with FLASK_APP=fhost flask prune. All files uploaded after the migration will have expiration times attached that the script will use to determine which files should be removed. However, files created prior to the migration will not have expirations associated. To address this, the script has an option (--legacy) which supplements the new-style exiration checking with the old-style expiration checking, which is sufficient to clean up legacy files as they expire.
The script at cleanup.py has been replaced with a deprecation notice informing users to switch to the new script, and to use the --legacy flag.
Information about how to add an expiration date to files has been added to the index page
The README has been updated to suggest users use the new cleanup script rather than the old cleanup script.
Two additional configuration parameters were added: FHOST_MIN_EXPIRATION and FHOST_MAX_EXPIRATION, each of which accept a duration in milliseconds, and parameterize the amount of time files last before expiring. Previously, these values were hardcoded in the cleanup script.

Please let me know what you think, and if there's any changes you think should be made!

Closes #33 This pull request adds support for an additional field during file uploads: `expires=<time>`. This field can be set to either: - A time in hours for which the file should be retained - A timestamp in epoch milliseconds at which the file should be expired Neither option allows setting an expiration time longer than what would naturally be allotted to the file. In other words, this allows users to shorten the retention time of their files, but not extend it. In order to accomadate this change, the following changes were necessary: 1. Adding an `expiration` field to the `file` table in the database. Prior to this change, expiration dates were tracked using filesystem metadata. Unfortunately, I do not know a way to use filesystem metadata to store another date/duration to expire the file at, so it was necessary to replace this method of tracking expirations with the database 2. The creation of a migration script to facilitate the above database change. 3. As a result of the above two changes, the `cleanup.py` script needed to be overhauled. Because it now needs database connectivity, I elected to move it into the main flask app as an alternate entrypoint, which can now be executed with `FLASK_APP=fhost flask prune`. All files uploaded after the migration will have expiration times attached that the script will use to determine which files should be removed. However, files created prior to the migration will not have expirations associated. To address this, the script has an option (`--legacy`) which supplements the new-style exiration checking with the old-style expiration checking, which is sufficient to clean up legacy files as they expire. 4. The script at `cleanup.py` has been replaced with a deprecation notice informing users to switch to the new script, and to use the `--legacy` flag. 5. Information about how to add an expiration date to files has been added to the index page 6. The README has been updated to suggest users use the new cleanup script rather than the old cleanup script. 7. Two additional configuration parameters were added: `FHOST_MIN_EXPIRATION` and `FHOST_MAX_EXPIRATION`, each of which accept a duration in milliseconds, and parameterize the amount of time files last before expiring. Previously, these values were hardcoded in the cleanup script. Please let me know what you think, and if there's any changes you think should be made!

Ember added 1 commit 2022-11-22 22:47:23 +01:00

Add support for expiring files 6d036eeb7d

SUPPLEMENTALLY:
- Add an `expiration` field to the `file` table of the database
- Produce a migration for the above change
- Overhaul the cleanup script, and integrate into fhost.py
  (now run using FLASK_APP=fhost flask prune)
- Replace the old cleanup script with a deprecation notice
- Add information about how to expire files to the index
- Update the README with information about the new script

Ember added 1 commit 2022-11-22 22:52:25 +01:00

Add a note explaining that expired files aren't immediately removed 78b8a73ea5

mia commented

2022-11-22 23:12:30 +01:00

Oh, nice work! Thank you!

But how about instead of recording the absolute expiration date, record both creation date + client-requested offset (if any) in the database? That allows changing the expiration settings (or the entire algorithm) at a later point, as well as more practical database queries. I actually want to stop relying on filesystem metadata.

Also, maybe populate the missing database fields during migration so we don’t need the --legacy flag?

Oh, nice work! Thank you! But how about instead of recording the absolute expiration date, record both creation date + client-requested offset (if any) in the database? That allows changing the expiration settings (or the entire algorithm) at a later point, as well as more practical database queries. I actually want to stop relying on filesystem metadata. Also, maybe populate the missing database fields during migration so we don’t need the `--legacy` flag?

mia added the

enhancement

label 2022-11-22 23:14:27 +01:00

mia changed title from ~~Support user-specified expiration times~~ to WIP: Support user-specified expiration times

2022-11-22 23:14:52 +01:00

Ember added 1 commit 2022-11-22 23:17:12 +01:00

Show correct times on the index page graph bed8c1d047

Ember commented

2022-11-22 23:21:23 +01:00

That makes sense!

I'll do my best to figure out populating database fields! I originally considered it, but got scared off because I wasn't entirely sure how to work with the database environment that alembic executes in (and without the sqlalchemy models), but I'll see if I can work it out!

That makes sense! I'll do my best to figure out populating database fields! I originally considered it, but got scared off because I wasn't entirely sure how to work with the database environment that alembic executes in (and without the sqlalchemy models), but I'll see if I can work it out!

👍 1

mia commented

2022-11-22 23:57:37 +01:00

Also the reupload case with existing files is kinda ugly and would allow bad actors to go around changing dates on random files. Some form of auth mechanism would be nice to have there, but I’d say that’s out of scope for this PR.

Not sure what to do in the meantime. How about this: If an expiration offset has been set already, do not modify the uploaded_at (or whatever you’d call it) timestamp or the offset, to enforce expiry at the time set by the original uploader. Only do that once the file has vanished.

Also the reupload case with existing files is kinda ugly and would allow bad actors to go around changing dates on random files. Some form of auth mechanism would be nice to have there, but I’d say that’s out of scope for this PR. Not sure what to do in the meantime. How about this: If an expiration offset has been set already, do not modify the `uploaded_at` (or whatever you’d call it) timestamp or the offset, to enforce expiry at the time set by the original uploader. Only do that once the file has vanished.

Ember commented

2022-11-23 00:18:28 +01:00

I'm not entirely sure I follow? Bad actors could theoretically extend expiration dates, but they wouldn't be able to shorten it, which is the same current behaviour anyway.

I guess if a user needs the file to be deleted after a certain point, that could be a problem, but I feel like 0x0 can't offer that guarantee of safety anyway, since files aren't even guaranteed to be deleted at their expirations, and can easily end up on archive.org or other sites anyway.

I'm not entirely sure I follow? Bad actors could theoretically extend expiration dates, but they wouldn't be able to shorten it, which is the same current behaviour anyway. I guess if a user *needs* the file to be deleted after a certain point, that could be a problem, but I feel like 0x0 can't offer that guarantee of safety anyway, since files aren't even guaranteed to be deleted at their expirations, and can easily end up on archive.org or other sites anyway.

mia commented

2022-11-23 00:23:12 +01:00

Hm fair enough. Might not be an issue then for this feature 🤷

Ember commented

2022-11-26 23:48:10 +01:00

Alright so I thought about this a little bit, and I think you were right that there's a dilemma with the reupload case. Specifically, I'm thinking of this case, where a re-upload comes shortly before a change in the expiration parameters:

T+00h - A client uploads a file that the server assigns a natural lifespan of >24h.  The client requests an expiration of 24h
T+12h - A different client uploads the same file, and requests a lifespan of 6h.
T+14h - The server changes the expiration parameters.  The same file is now regarded as having a natural lifespan of 8h.

I can think of four different behaviours the server could respond with:

Behavior 1

At T+00h, the file has a calculated expiration of T+24h (requested). When the file is re-uploaded, the file still has an expiration of T+24h, since the new requested expiration (T+18h) is earlier than the old requested expiration, however the new upload time and requested expiration are still stored. At T+14h, when the expiration parameters change, the server recognizes that no more time is owed to the first upload, and the later upload's requested expiration is used, so that the calculated expiration is T+18h.

Implementation aside, I think this is mathematically ideal behaviour for a system which allows retroactive application of expiration parameters. It ensures that files are expired exactly as soon as is permissible. However, in order to implement it, we'd need to store a full history of upload+requested expiration timestamps, as opposed to just the most recent, which would also require significant restructuring of the database, since this would violate existing uniqueness constraints.

Main Benefit: When expiration parameters change, they can be retroactively applied.
Main Downside: Faster database growth, even when expiration parameters are constant.

Behavior 2

At T+00h, the file has a calculated expiration of T+24h. When the file is re-uploaded, the server observes that the requested expiration is before the existing expiration. The server updates the most recent upload date of file to T+12h, but does not change the expiration date from T+24h (or equivalently, changes the requested duration to 12h, preserving the T+24h expiration date). When the server changes the expiration parameters, the server observes that the natural expiration (now T+20h thanks to the updated upload time) is before the recorded requested expiration (T+24h). The file expires at T+20h.

I see this behavior as a compromise of behavior 1. It preserves the ability to retroactively apply expiration parameters and extend expiration dates without needing to track historic upload times at the cost of efficiency. Specifically, this file could be culled at T+18h, but the algorithm protects it until T+20h.

As far as database requirements, this requires storing a minimum of two timestamps - most recent upload and requested expiration

Main Benefit: When expiration parameters change, they can be retroactively applied.
Main Downside: When expiration parameters change, new calculated expirations can be inaccurate (too short or too long)

Behavior 3

When the file is uploaded, the uploader is guaranteed an expiration no sooner than T+24h. When the file is re-uploaded, the server observes that the requested expiration is sooner than the current expiration, and the existing expiration is preserved. This client is also guaranteed an expiration no sooner than T+24h. When the server changes expiration parameters, they are not retroactively applied to existing files. The file expires at T+24h.

This behavior sacrifices retroactive application of expiration parameters, and in exchange, is able to guarantee a file's minimum lifetime at upload time. This is the behavior as it is currently implemented in this PR. It requires storing a minimum of one timestamp - the latest guaranteed expiration. I'll also admit a bit of a bias on my part towards this solution, because the stable expiration time is something I like.

Main Benefit: Expiration times are stable and minimum lifetimes can be guaranteed at upload time.
Main Downside: No retroactive application of expiration parameters.

Behavior 4

When the file is uploaded, the calculated expiration is T+24h, and re-uploads are not permitted (maybe only until file expiration?). When the file is re-uploaded, the request is simply denied, either with a no-op 200 Okay or a 4XX. When the expiration parameters are changed, they apply to the file, and it expires immediately at T+14h.

This is my interpretation of your above comment, although I may have misinterpreted you? It requires storing two timestamps - original upload and original requested expiration. I'm personally not sure that this use case is something we should / are able to support for the reasons I described above, but I figured it was still worth mentioning as a possible behavior.

Main Benefit: File lifetimes cannot be extended beyond their original requested expiration
Main Downside: File lifetimes cannot be extended beyond their original requested expiration

Discussion

One thing I think that's worth noting is that none of this behavior applies except in a bit of an edge case where a file is being reuploaded shortly before a change in the expiration parameters. Obviously this only affects a very very small number of users and servers, so it's not a very consequential decision, although it will considerably affect how this PR is developed.

The one exception to this is Behavior 4, which would impact expiration behavior outside of this edgecase.

Honestly, I wouldn't be writing such a long comment in discussion of this if it weren't for the fact that I'm currently trapped on a 12h train trip without WiFi or reception, and nothing to do but read PL papers and type excruciatingly long comments in .txt files. That said, I think I'm out of steam now anyway, so I'll wrap it up.

Like I said, my personal preference is behavior three, but I'm willing to implement any of these that you deem best for the project. Lemme know what you think, and thanks for tolerating my boredom-induced overthinking!

Alright so I thought about this a little bit, and I think you were right that there's a dilemma with the reupload case. Specifically, I'm thinking of this case, where a re-upload comes shortly before a change in the expiration parameters: ``` T+00h - A client uploads a file that the server assigns a natural lifespan of >24h. The client requests an expiration of 24h T+12h - A different client uploads the same file, and requests a lifespan of 6h. T+14h - The server changes the expiration parameters. The same file is now regarded as having a natural lifespan of 8h. ``` I can think of four different behaviours the server could respond with: ### Behavior 1 At T+00h, the file has a calculated expiration of T+24h (requested). When the file is re-uploaded, the file still has an expiration of T+24h, since the new requested expiration (T+18h) is earlier than the old requested expiration, however the new upload time and requested expiration are still stored. At T+14h, when the expiration parameters change, the server recognizes that no more time is owed to the first upload, and the later upload's requested expiration is used, so that the calculated expiration is T+18h. Implementation aside, I think this is mathematically ideal behaviour for a system which allows retroactive application of expiration parameters. It ensures that files are expired exactly as soon as is permissible. However, in order to implement it, we'd need to store a full history of upload+requested expiration timestamps, as opposed to just the most recent, which would also require significant restructuring of the database, since this would violate existing uniqueness constraints. - **Main Benefit:** When expiration parameters change, they can be retroactively applied. - **Main Downside:** Faster database growth, even when expiration parameters are constant. ### Behavior 2 At T+00h, the file has a calculated expiration of T+24h. When the file is re-uploaded, the server observes that the requested expiration is before the existing expiration. The server updates the most recent upload date of file to T+12h, but does not change the expiration date from T+24h (or equivalently, changes the requested duration to 12h, preserving the T+24h expiration date). When the server changes the expiration parameters, the server observes that the natural expiration (now T+20h thanks to the updated upload time) is before the recorded requested expiration (T+24h). The file expires at T+20h. I see this behavior as a compromise of behavior 1. It preserves the ability to retroactively apply expiration parameters and extend expiration dates without needing to track historic upload times at the cost of efficiency. Specifically, this file could be culled at T+18h, but the algorithm protects it until T+20h. As far as database requirements, this requires storing a minimum of two timestamps - most recent upload and requested expiration - **Main Benefit:** When expiration parameters change, they can be retroactively applied. - **Main Downside:** When expiration parameters change, new calculated expirations can be inaccurate (too short or too long) ### Behavior 3 When the file is uploaded, the uploader is guaranteed an expiration no sooner than T+24h. When the file is re-uploaded, the server observes that the requested expiration is sooner than the current expiration, and the existing expiration is preserved. This client is also guaranteed an expiration no sooner than T+24h. When the server changes expiration parameters, they are not retroactively applied to existing files. The file expires at T+24h. This behavior sacrifices retroactive application of expiration parameters, and in exchange, is able to guarantee a file's minimum lifetime at upload time. This is the behavior as it is currently implemented in this PR. It requires storing a minimum of one timestamp - the latest guaranteed expiration. I'll also admit a bit of a bias on my part towards this solution, because the stable expiration time is something I like. - **Main Benefit:** Expiration times are stable and minimum lifetimes can be guaranteed at upload time. - **Main Downside:** No retroactive application of expiration parameters. ### Behavior 4 When the file is uploaded, the calculated expiration is T+24h, and re-uploads are not permitted (maybe only until file expiration?). When the file is re-uploaded, the request is simply denied, either with a no-op `200 Okay` or a `4XX`. When the expiration parameters are changed, they apply to the file, and it expires immediately at T+14h. This is my interpretation of your above comment, although I may have misinterpreted you? It requires storing two timestamps - original upload and original requested expiration. I'm personally not sure that this use case is something we should / are able to support for the reasons I described above, but I figured it was still worth mentioning as a possible behavior. - **Main Benefit:** File lifetimes cannot be extended beyond their original requested expiration - **Main Downside:** File lifetimes cannot be extended beyond their original requested expiration ## Discussion One thing I think that's worth noting is that none of this behavior applies except in a bit of an edge case where a file is being reuploaded shortly before a change in the expiration parameters. Obviously this only affects a very very small number of users and servers, so it's not a very consequential decision, although it will considerably affect how this PR is developed. The one exception to this is Behavior 4, which would impact expiration behavior outside of this edgecase. Honestly, I wouldn't be writing such a long comment in discussion of this if it weren't for the fact that I'm currently trapped on a 12h train trip without WiFi or reception, and nothing to do but read PL papers and type excruciatingly long comments in .txt files. That said, I think I'm out of steam now anyway, so I'll wrap it up. Like I said, my personal preference is behavior three, but I'm willing to implement any of these that you deem best for the project. Lemme know what you think, and thanks for tolerating my boredom-induced overthinking!

Ember added 1 commit 2022-11-27 03:24:29 +01:00

Improve the migration script, removing the need for --legacy f507de8256

Ember commented

2022-11-27 03:36:13 +01:00

f507de8 adds support for calculating an expiration date for files during the migration. It's quite a bit jenky and I think it might violate some of the principles of alembic's migration system, but I couldn't really figure out a better way of doing it, although please let me know if you do.

Still on the TODO:

Pick a re-upload behavior (as above)
Implement the db schema changes based on that
???

f507de8 adds support for calculating an expiration date for files during the migration. It's quite a bit jenky and I think it might violate some of the principles of alembic's migration system, but I couldn't really figure out a better way of doing it, although please let me know if you do. Still on the TODO: - Pick a re-upload behavior (as above) - Implement the db schema changes based on that - ???

mia commented

2022-11-28 11:04:33 +01:00

I think behavior 3 is fine. Changing the expiration logic is more of an edge case after all. Thank you for putting this much thought into it!

I’ve taken a look at the migration script and added some review comments. Let me know what you think.

I think behavior 3 is fine. Changing the expiration logic is more of an edge case after all. Thank you for putting this much thought into it! I’ve taken a look at the migration script and added some review comments. Let me know what you think.

mia requested changes 2022-11-28 11:05:58 +01:00

fhost.py

					
				@ -367,0 +463,4 @@

				                File.expiration.is_not(None),

				                File.expiration < current_time

				            )

				        ).all()

mia commented

2022-11-28 11:00:36 +01:00

.all() can be omitted here—unless you want to add a fancy progress bar with tqdm?

`.all()` can be omitted here—unless you want to add a fancy progress bar with `tqdm`?

Ember commented

2022-11-28 17:46:47 +01:00

Noted! Will do! I think I'll leave out the fancy progress bar because I expect that this script (or at least the part of it that does more than add a new column to an empty database) will run a scarce few times, and there's really only a few existing servers where I expect it to take more than a couple seconds.

Ember commented

2022-11-28 22:48:24 +01:00

Oh wait I got mixed up, and thought this comment was on the migration script. Still, I doubt it'll take very long to run, even on larger servers.

Ember commented

2022-11-28 22:49:47 +01:00

11cfd07

11cfd07

Ember marked this conversation as resolved

migrations/versions/939a08e1d6e5_.py Outdated

					
				@ -0,0 +38,4 @@

				db = SQLAlchemy(current_app.__weakref__())

				# Representations of the original and updated File tables

				class File(db.Model):

mia commented

2022-11-28 10:33:13 +01:00

It might be possible to use SQLAlchemy’s automap extension to reflect the schema as it exists. A minimal example I have not tested:

import sqlalchemy as sa
from alembic import op
from sqlalchemy.ext.automap import automap_base

Base = automap_base()

def upgrade():
    op.add_column('file', sa.Column('expiration', sa.BigInteger()))

    bind = op.get_bind()
    Base.prepare(autoload_with=bind)
    File = Base.classes.file

It might be possible to use SQLAlchemy’s [automap](https://docs.sqlalchemy.org/en/14/orm/extensions/automap.html) extension to reflect the schema as it exists. A minimal example I have not tested: ```python import sqlalchemy as sa from alembic import op from sqlalchemy.ext.automap import automap_base Base = automap_base() def upgrade(): op.add_column('file', sa.Column('expiration', sa.BigInteger())) bind = op.get_bind() Base.prepare(autoload_with=bind) File = Base.classes.file ```

Ember commented

2022-11-28 17:47:07 +01:00

Ooooooo! I hadn't heard of that before! Thanks for bringing it up, I'll take a look!

Ember commented

2022-11-28 22:19:02 +01:00

Resolved in d14713d

Resolved in d14713d

Ember marked this conversation as resolved

migrations/versions/939a08e1d6e5_.py Outdated

					
				@ -0,0 +65,4 @@

				    # Calculate an expiration date for all existing files

				    files = File.query\

				        .where(

				            sa.not_(File.removed)

mia commented

2022-11-28 10:13:31 +01:00

You can change this query to: File.query.where(sa.not_(File.removed), File.sha256.in_(unexpired_files)), also omitting the .all(). This is much faster on my instance, which has over a million results for the unmodified query. That’s a lot of objects we don’t need!

You can change this query to: `File.query.where(sa.not_(File.removed), File.sha256.in_(unexpired_files))`, also omitting the `.all()`. This is much faster on my instance, which has over a million results for the unmodified query. That’s a *lot* of objects we don’t need!

Ember commented

2022-11-28 17:48:50 +01:00

I deliberately chose not to do that here because I'm worried that the query will become too large, and I wasn't sure whether or not sqlalchemy has provisions built in to handle that case.

I've had trouble before with very large in y queries when working directly with the sqlite library, and I didn't want to chance it. If you think it's safe though, I'll make the change.

I deliberately chose not to do that here because I'm worried that the query will become too large, and I wasn't sure whether or not sqlalchemy has provisions built in to handle that case. I've had trouble before with very large `in y` queries when working directly with the sqlite library, and I didn't want to chance it. If you think it's safe though, I'll make the change.

mia commented

2022-11-28 20:06:10 +01:00

I think that should be safe unless the SQL statement somehow grows larger than a gigabyte (SQLite’s default limit), which seems unlikely.

👍 1

Ember commented

2022-11-28 22:25:09 +01:00

Resolved in 19d989b

Resolved in 19d989b

Ember marked this conversation as resolved

migrations/versions/939a08e1d6e5_.py Outdated

					
				@ -0,0 +73,4 @@

				            stat = os.stat(file_path)

				            max_age = get_max_lifespan(stat.st_size) # How long the file is allowed to live, in ms

				            file_birth = stat.st_mtime * 1000 # When the file was created, in ms

				            op.execute(

mia commented

2022-11-28 10:43:24 +01:00

Instead of doing this here, it’s probably faster to store the modified fields:

# assume updates is a list
updates.append({
    "id": file.id,
    "expiration": int(file_birth + max_age)
})

and execute a bulk update at the end:

# we’ll have to attach an SQLAlchemy session
from sqlalchemy.orm.session import Session

# reuse bind from automap example if applicable
session = Session(bind=op.get_bind())
# UpdatedFile if not using the automapped class
session.bulk_update_mappings(File, updates)
session.commit()

Instead of doing this here, it’s probably faster to store the modified fields: ```python # assume updates is a list updates.append({ "id": file.id, "expiration": int(file_birth + max_age) }) ``` and execute a bulk update at the end: ```python # we’ll have to attach an SQLAlchemy session from sqlalchemy.orm.session import Session # reuse bind from automap example if applicable session = Session(bind=op.get_bind()) # UpdatedFile if not using the automapped class session.bulk_update_mappings(File, updates) session.commit() ```

Ember commented

2022-11-28 17:52:48 +01:00

Makes sense! I'll implement it when I get out of classes later today.

Ember commented

2022-11-28 22:29:07 +01:00

Resolved in 55ee374

Resolved in 55ee374

Ember marked this conversation as resolved

Ember added 2 commits 2022-11-28 22:18:59 +01:00

Use automap in place of an explicit file map in migration d14713d077

Remove vestigial touch() 39d24e56c3

Ember added 3 commits 2022-11-28 22:25:03 +01:00

Don't crash when upgrading a fresh database 64ddfdb702

Remove vestigial warning about legacy files 74b6f986ee

More efficiently filter to unexpired files when migrating 19d989b696

#72 (comment)

Ember added 1 commit 2022-11-28 22:28:56 +01:00

Coalesce updates to the database during migration 55ee3740b0

#72 (comment)

Ember commented

2022-11-28 22:29:40 +01:00

How does that look?

Ember requested review from mia 2022-11-28 22:29:46 +01:00

mia reviewed 2022-11-28 22:41:00 +01:00

migrations/versions/939a08e1d6e5_.py Outdated

					
				@ -0,0 +40,4 @@

				db = SQLAlchemy(current_app.__weakref__())

				# Representation of the updated (future) File table

				UpdatedFile = sa.table('file',

mia commented

2022-11-28 22:41:00 +01:00

Is this still needed?

Ember commented

2022-11-28 22:44:11 +01:00

Nope! Missed that!

60db793

Nope! Missed that! 60db793

Ember marked this conversation as resolved

mia commented

2022-11-28 22:43:41 +01:00

Starting to look good! Think I’ll test this in a bit.

Ember added 1 commit 2022-11-28 22:44:06 +01:00

Remove vestigial database model 60db7938c8

#72 (comment)

Ember added 1 commit 2022-11-28 22:49:39 +01:00

prune: Stream expired files from the database 11cfd07d71

(as opposed to collecting them all first)

mia reviewed 2022-11-28 22:59:07 +01:00

migrations/versions/939a08e1d6e5_.py

					
				@ -0,0 +31,4 @@

				Value returned is a duration in milliseconds.

				"""

				def get_max_lifespan(filesize: int) -> int:

mia commented

2022-11-28 22:59:07 +01:00

Note to self: Move utility functions like these to separate files…

mia approved these changes 2022-11-28 22:59:20 +01:00

Ember changed title from ~~WIP: Support user-specified expiration times~~ to Support user-specified expiration times

2022-11-28 23:05:05 +01:00

Ember added 2 commits 2022-11-28 23:09:40 +01:00

Merge remote-tracking branch 'mia/master' into expiration b6833b6e51

config.example.py: Add min & max expiration + description 7bb18b17dc

mia commented

2022-11-29 16:16:04 +01:00

Alright, did a quick test run and it seems to be working as expected. Squashed and merged as af4b3b06c0. Thank you so much!

Alright, did a quick test run and it seems to be working as expected. Squashed and merged as af4b3b06c0. Thank you so much!

❤️ 1

mia closed this pull request

2022-11-29 16:16:04 +01:00

mia reopened this pull request

2022-11-29 16:16:20 +01:00

mia manually merged commit af4b3b06c0 into master

2022-11-29 16:20:08 +01:00

Sign in to join this conversation.

No reviewers

No labels

No milestone

No project

No assignees

2 participants

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: mia/0x0#72