Load simulator #3125
Draft
Load simulator #3125
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add rems.simulate and rems.simulate-util namespaces. Load simulator works by spawning threads that operate indefinitely on separate headless chrome instances. Each thread then run actions against supplied REMS URL. Thread daemon wakes up periodically to check thread status and restarts if necessary. Threads choose random users and actions.
rems.simulate imports both rems.test-browser and rems.browser-test-util namespaces, which imports rems.main, which imports rems.simulate, causing circular reference.
Tests may call (mount/start) which inadvertently could start up simulator if namespace was loaded, because arguments were not validated.
aatkin
force-pushed
the
anssi/load-simulator
branch
from
f039ba0
to
4059403
Compare
Only parse CLI opts when args are passed from CLI, and support passing args from REPL as regular map.
Threads could read stale data when fetching new user and use same user. Technically not a problem, but underlying mechanism was designed to assign unique users to each action. (locking) should achieve this, together with swapping atom value. More statistics are now reported, and more frequently. Task logging gets very noisy with many threads, so it now has debug as default log level.
Experimental namespace is probably better suited for load simulator, since it's not exactly testing but more an experimental tool to figure out REMS limits.
Namespace rems.experimental.load-simulator uses test-browser and browser-test-util which depend on test sources and dependencies. Since browser test utilities can be desirable outside of test context, it seems less enticing to try and split specific functionalities so that test dependencies or sources would not leak into build. Including these files and dependencies should not cause any issues in build.
aatkin
force-pushed
the
anssi/load-simulator
branch
from
d13cf0e
to
0a9e75d
Compare
Macroz
reviewed
May 9, 2023
| @@ -0,0 +1,172 @@ | |||
| (ns rems.experimental.load-simulator | |||
There was a problem hiding this comment.
I think this doesn't need to be marked as experimental. Maybe simulator package? Or even load-simulator if there won't be others. It's perhaps also not a traditional load testing (because it's rather heavy, uses real browser etc.) but definitely it is a "user simulator".
| @@ -498,7 +498,7 @@ | |||
| sort | |||
| rest)) | |||
|
|
|||
| (defn- random-long-string [& [n]] | |||
| (defn random-long-string [& [n]] | |||
There was a problem hiding this comment.
We could probably move this to some util?
| (defn get-task-user! [task-id] | ||
| (locking current-tasks | ||
| (let [current-users (map :user-id (vals @current-tasks)) | ||
| available-users (-> (simu/get-all-users) |
There was a problem hiding this comment.
On my machine it tries to use perf tester users which are not present on the login screen. I think overall we must still figure out a solution for specifying the users.
Some users might not exist in test data (e.g. perf users), so using direct login link should be more robust for simulation tasks.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Initial version of load simulator. Run concurrent headless chrome instances against target REMS. CLI accepts following optional arguments:
--url- which url to run against (default: http://localhost:3000/)--concurrency- how many threads to start (default: 8)Quick start:
lein run)lein run load-simulator [--url url] [--concurrency n](no args uses default values)Checklist for author
Remove items that aren't applicable, check items that are done.
Documentation
Testing
Follow-up