Queue: Better management of the timeout

Fix versions

Priority

Description

If Jira is slow, then Confluence times-out and believes the messages weren't treated, and retries them 3 minutes later.

Documentation

See this page for all explanations about our changes:

https://documentation.play-sql.com/display/RY/Queue+performance

Starving the connection pool

In Jira, restrict the message receival to 1 worker. This ensures we don’t starve the connection pool with n workers.

Timeouts

Ensure the emitter’s timeout is much longer than the recipient (4x30s),

Ensure the recipient hastily stops processing messages upon reaching the timeout (30s), returning them as NOT_EXECUTED, so the other messages are correctly marked as DONE in the emitter.

It still leaves a gap: If Jira takes 40s to respond every time, and the network cuts the connection at 30s every time, then Confluence is not aware that the messages have been processed, and will send the same messages next time. There is no workaround for this gap.

A possible solution would be to stream the results back to Confluence (same as long-polling methods), but this will introduce another class of problems.

Other

Don’t forget https://playsql.atlassian.net/browse/RY-751 .

Activity

Show:
Resolved

Details

Assignee

Reporter

Requirement Yogi

Created March 23, 2021 at 10:01 AM
Updated December 21, 2021 at 11:37 AM
Resolved March 28, 2021 at 2:15 PM