Performance

Summary

Basically, once you have 50.000 requirements in the database, expect 20ms per requirement on the page when you save a page, and 20ms per requirement on view. This is not a commitment, as it depends on the machine, the set up, the configuration, the version of Confluence and Requirement Yogi.

Performance improvements in v1.11.5

• For pages with no requirements, we've improved the speed by skipping our indexation:
  • We skip the parsing if the storage format hasn't changed,
  • We skip the parsing if the rendered format hasn't changed, in case it contains an "Include" or "Scaffolding" macro.
  • We skip the parsing if there is no requirement in the old or new version.
• For pages with requirements:
  • We've added indexes on database columns. On our instance we get 5x faster results when saving a page, but we may be in special circumstances.
  • When we index a page (=when a user saves a page), we've batched the lookups of requirements, so we don't do 1 database request for each requirement on the page. On our instance, we get again 4x faster times depending on database latency (most LANs are on 1ms latency, but we've measured with 5ms).
  • We'd be thrilled if you have 20x better response times than in 1.11.4, but we'll check back with customers before asserting that.

Performance change in 2.0

• We've deeply modified the indexing algorithm in 2.0, because we are now importing Excel files.
• This algo generally reads first and checks whether it needs to change the data, instead of deleting all and writing blindly.
• We did not notice much change in speed. Some speeds are improved, other are worse, depending on the number of modified requirements and properties.

Details

We have evaluated on a personal machine with the following setup:

• iMac 2013 – 3.4 GHz Intel Core i5,
• RAM 8GB.
• Database latency: 5 to 10ms (random) per query.
• Network latency to the database: 2ms (simulated) per query,
• Database prefilled with 80.000 requirements.

We have simply instrumented the code and created massive pages:

Errors in the logs?

If your server meets problems like "OutOfMemoryException", "Java Heap Space" or "SiteMesh" exceptions, it could be related to building the Traceability matrix. One important thing to note is that the Requirement Yogi add-on may not be mentioned in those exceptions. If you are in this situation:

• Use queries that return fewer requirements, in the search, but more importantly in the dependency, traceability and coverage matrix.
• The dependency matrix and the coverage report can only be built by users who can export the space (This permission will change in version 2.0, see Release Notes 2.0).
• Therefore, inform a subset of your users about the server limits and the tips to prevent a memory overflow; and restrict the export permissions to this group only.
• As a last resort, don't use the coverage and dependency matrixes. You can always explore Confluence's database schema and run the queries using an SQL tool. Tables related to Requirement Yogi start with "AO_32F7CE_AOREQUIREMENT".

Also, please notify us if you're meeting performance issues. We've obtained Data Center certification with several performance tests, but real-life environment can be different than prepared tests and we want to keep improving.

Performance of upgrade tasks

Upgrade tasks are executed once, when the addon is upgraded to the provided version.