Use workerqueue for mailstream jobs instead of custom table #1133

Merged

mexon merged 2 commits from mat/mailstream-workerqueue into 2021.06-rc

2021-06-16 23:57:07 +02:00

mexon commented

2021-06-10 18:16:45 +02:00

(Migrated from github.com)

This uses a completely different approach to managing retries, that avoids the need to create a whole table just for this plugin.

The main reason I need retries is to work nicely with retriever (which I will eventually submit, but I still need to make it use the workerqueue). Retriever marks items as invisible until it's finished processing them, and they shouldn't be mailed until then. Mailstream will now retry every hour until it succeeds. I'm hoping this doesn't cause too many problems.

This uses a completely different approach to managing retries, that avoids the need to create a whole table just for this plugin. The main reason I need retries is to work nicely with retriever (which I will eventually submit, but I still need to make it use the workerqueue). Retriever marks items as invisible until it's finished processing them, and they shouldn't be mailed until then. Mailstream will now retry every hour until it succeeds. I'm hoping this doesn't cause too many problems.

annando (Migrated from github.com) reviewed 2021-06-10 18:39:43 +02:00

annando commented

2021-06-12 21:39:52 +02:00

(Migrated from github.com)

Have you tested it on your system?

mexon commented

2021-06-12 21:48:24 +02:00

(Migrated from github.com)

Have you tested it on your system?

I'm running it on my test system, with a patch that makes mail sending randomly fail 7/8 of the time to properly exercise the code. The test system generates a post every 20 minutes, and I'm watching the emails arrive as well as watching the retries in the workerqueue. It seems pretty robust, all the emails do arrive eventually.

I haven't put it on my live system yet. The fun part is the conversion of the existing custom table when you upgrade. Before I upgrade my live system I'd have to make a site backup, and since the database is quite large I've been avoiding that. Would you like me to run it on live as well before accepting the PR?

I'm not quite sure why the build above is failing?

> Have you tested it on your system? I'm running it on my test system, with a patch that makes mail sending randomly fail 7/8 of the time to properly exercise the code. The test system generates a post every 20 minutes, and I'm watching the emails arrive as well as watching the retries in the workerqueue. It seems pretty robust, all the emails do arrive eventually. I haven't put it on my live system yet. The fun part is the conversion of the existing custom table when you upgrade. Before I upgrade my live system I'd have to make a site backup, and since the database is quite large I've been avoiding that. Would you like me to run it on live as well before accepting the PR? I'm not quite sure why the build above is failing?

annando commented

2021-06-12 22:20:55 +02:00

(Migrated from github.com)

@nupplaphil do you know why it fails?