Adding an EMS Operation Mode and its Configuration Settings
This guide is the checklist for adding a new EMS operational mode together with its settings page. It is written from the SELF_TEST mode, which was added in exactly this way, so every step below has a working reference implementation you can copy.
Read operation-modes.md first for what an operation
mode is and how the control loop dispatches it.
1. What adding a mode actually touches
A mode is not one file. It spans four layers, and a mode that is registered in only some of them fails in a specific, confusing way:
| Layer | File | If you forget it |
|---|---|---|
| Mode registry | ems_operation_modes.py |
The mode is rejected and silently replaced by IDLE when it is written to the settings, so the EMS appears to accept the mode and then runs the plant as IDLE. |
| Dispatcher | ems_main.py |
The loop falls through to a no-op or the previous branch; the mode runs but does nothing. |
| Settings structure | ems_control_setting.template.json (tracked) and the runtime emsControl/ems/ems_control_setting.json |
The parameters the mode reads do not exist, so the mode uses hardcoded defaults. |
| Settings page | emsConfigurationOperation.blade.php, cruds.php, EmsSettingTableSeeder.php |
An operator cannot see or change the parameters from the UI. |
The mode string is the join between all four layers. Pick it once
(SCREAMING_SNAKE_CASE, e.g. SELF_TEST), keep the function block name in
PascalCase (Self_Test), and use the kebab-case form (self-test) for the tab id.
2. How the settings page works
Before editing the page it helps to know that almost nothing about it is
per-mode. public/js/bess-config-mapper.js drives every tab generically:
- A tab is three pieces of markup, with no JavaScript to write:
<li class="tab" data-tab="<tab-id>">in the<ul class="tabs">strip.<div class="tab-pane" id="<tab-id>">holding the inputs.<button class="save-tab-btn" data-tab="<tab-id>">inside a<div class="tab-save-section">.- Inputs bind to the settings JSON with
data-json-path. On load the mapper reads each path out of the config and fills the input; on Save it walks every[data-json-path]inside that tab pane and writes the values back. The path is the literal JSON path, e.g.functions.Self_Test.alarm_diagnostic_test.appear_timeout_s. - Empty inputs are skipped by default.
getValueFromInput()returnsnullfor an empty box and the mapper then leaves the old value in place, so a field cannot be cleared. For a field that must be clearable, adddata-json-emptyto say what empty means, e.g.data-json-empty=""for a list of strings. This attribute is why the SELF_TEST allow-list can be emptied from the UI. - Checkboxes are switches:
<label class="switch">with an<input type="checkbox">; the value is read fromchecked. - Number inputs are
type="number"; an empty box means "keep the old value", so always ship a sane default in the seeder. - Tabs can be permission gated with
@can('Some_Permission')around the<li>. The Diagnostic tab is ungated; most others are gated.
The configuration round trip is:
ems_settings.scada_settings_json (DB)
│ page load
▼
this.config in bess-config-mapper.js ──► inputs
▲ │ Save (save-tab-btn)
│ these paths only ▼
└────────────────────────── this.config updated in place
│ main Save button
▼
ems_settings row ──► EMS pulls it
▼
emsControl/ems/ems_control_setting.json
3. Step-by-step
Step 1 — Register the mode (ems_operation_modes.py)
EMS_OP_SELF_TEST = "SELF_TEST"
...
ALL_EMS_OPERATION_MODES = (
EMS_OP_IDLE,
...
EMS_OP_SELF_TEST, # add it to the tuple
)
Registering it is what makes the value survive validation. Skip this and the mode string is discarded in favour of IDLE.
Step 2 — Dispatch it (ems_main.py)
Add a branch in the control loop. If the mode must not run plant control, this is where you guarantee that:
elif mode == ems_operation_modes.EMS_OP_SELF_TEST:
# SELF TEST runs no plant control: the alarm diagnostic test drives
# the registers, and a concurrent dispatch would fight it.
pass
Step 3 — Add the settings structure
Add the function block to the tracked template, which is the versioned definition of the settings shape:
EnergySyc_EMS_ES100Cx12S/emsConfig/ems/ems_control_setting.template.json
"functions": {
"Self_Test": {
"enabled": true,
"description": "Reserves the EMS for the full alarm diagnostic test. ...",
"alarm_diagnostic_test": {
"include_self_test_alarms": true,
"appear_timeout_s": 30,
"clear_timeout_s": 45,
"row_settle_s": 8,
"max_alarms_per_device": 0,
"simulator_url": null,
"allow_simulator_register_writes": []
}
},
Note the runtime file emsControl/ems/ems_control_setting.json is gitignored
on purpose - its comment says the key structure is versioned in
ems_control_setting.template.json instead. Only edit the runtime file on a
live host, and never commit it.
Step 4 — Add the tab to the settings page
In resources/views/emsConfigurationOperation.blade.php, add the three pieces of
markup. Keep the tab next to the related ones:
{ 1. the tab button, inside <ul class="tabs"> }
<li class="tab" data-tab="self-test">
<i class="fas fa-vial"></i>
<span>{{ trans('cruds.emsSettings.Self_Test') }}</span>
</li>
{ 2. the pane, inside <div class="tab-content"> }
<div class="tab-pane" id="self-test">
<h5 class="tab-title">{{ trans('cruds.emsSettings.Self_Test_Settings') }} </h5>
<div class="settings-grid">
<div class="row">
<div class="col-md-6">
<div class="card">
<h6><i class="fas fa-vial"></i>{{ trans('cruds.emsSettings.Self_Test_Mode') }} </h6>
<div class="form-group">
<label>{{ trans('cruds.emsSettings.Appear_Timeout') }} </label>
<input type="number" step="1" min="1"
data-json-path="functions.Self_Test.alarm_diagnostic_test.appear_timeout_s">
<span class="field-info">{{ trans('cruds.emsSettings.Appear_Timeout_Info') }} </span>
</div>
</div>
</div>
</div>
</div>
{ 3. the save button }
<div class="tab-save-section">
<button class="btn save-tab-btn" data-tab="self-test" id="saveSelfTestSettings">
<i class="fas fa-save"></i> {{ trans('cruds.emsSettings.Save_Self_Test') }}
</button>
</div>
</div>
A clearable field carries data-json-empty:
<textarea rows="3" data-json-empty=""
data-json-path="functions.Self_Test.alarm_diagnostic_test.allow_simulator_register_writes"></textarea>
Step 5 — Add the labels (resources/lang/en/cruds.php)
Every trans('cruds.emsSettings.X') you used needs a key in the emsSettings
array, otherwise the page shows the raw key:
'Self_Test' => 'Self Test',
'Self_Test_Settings' => 'Self Test Settings',
'Save_Self_Test' => 'Save Self Test Settings',
Step 6 — Add the block to the seeder (database/seeders/EmsSettingTableSeeder.php)
The seeder holds the whole settings document as scada_settings_json. Add the
same function block there.
Keep the already selected mode untouched:
'ems_operation_mode' => 'MANUAL',
'ui_state' => [ 'radio_selected_function' => 'MANUAL', ... ],
A plant must never boot into a mode that suspends control. The new mode is selectable, not selected.
Step 7 — Optional: add the tab to the debug map
public/js/bess-config-mapper.js, getTabConfig() only feeds the console log;
add it for consistency:
'self-test': this.config.functions.Self_Test,
Step 8 — Deploy and verify
# Always run artisan as the web user. A root run in this container leaves
# root-owned files under storage/ and bootstrap/cache, which php-fpm (www-data)
# can read but not overwrite: every later view recompile then fails with
# "file_put_contents(...): Permission denied" and returns 500 to real users.
#
# route:clear is what makes a NEW route visible at all, because
# bootstrap/cache/routes-v7.php is authoritative at runtime.
docker exec -u www-data ems-frontend php artisan route:clear
docker exec -u www-data ems-frontend php artisan view:clear
docker exec -u www-data ems-frontend php artisan config:clear
Then confirm, in this order:
- Renders - the tab appears and shows the values, not the raw
transkeys. - Saves - change a value, Save, and reload; the value persists.
- Persists - the DB row carries it:
SELECT JSON_EXTRACT(scada_settings_json, '$.functions.Self_Test') FROM ems_settings; - Reaches the EMS - after the settings pull, the live file matches:
docker exec ems-backend python3 -c "import json;print(json.load(open('/app/EnergySyc_EMS_ES100Cx12S/emsControl/ems/ems_control_setting.json'))['functions']['Self_Test'])" - Mode is accepted - select it and confirm the EMS reports it back rather than falling back to IDLE. If it falls back, Step 1 is missing.
- Mode does nothing harmful - in the new mode the plant must be idle.
4. SELF_TEST settings reference
functions.Self_Test.enabled — shows SELF TEST in the Operational Functions
selector in the Operation tab.
functions.Self_Test.alarm_diagnostic_test.include_self_test_alarms — also
exercise the SELFTEST-nnn alarms, which are injected without touching any
device.
functions.Self_Test.alarm_diagnostic_test.max_alarms_per_device — 0 tests
every configured alarm of the device; any other value caps the run.
functions.Self_Test.alarm_diagnostic_test.appear_timeout_s /
clear_timeout_s — how long to wait for the triggered alarm to appear, and for
it to disappear after the trigger is reverted, before recording a timeout.
functions.Self_Test.alarm_diagnostic_test.row_settle_s — delay before reading
active_alarm_records, so the 1 Hz collector has written its row.
functions.Self_Test.alarm_diagnostic_test.simulator_url — base URL of the
simulator register API, e.g. http://bess-simulator-backend:8000. Empty means no
device register is ever written.
functions.Self_Test.alarm_diagnostic_test.allow_simulator_register_writes —
the device_type:field pairs the test may write, e.g.
grid_energy_meter:frequency_hz. Accepts a JSON list or the comma separated
string the settings page writes. The second element is the rule field from
the device alarm map (frequency_hz), not the reading key
(gridMeter_frequency_hz).
Safety: three independent opt-ins
A diagnostic test only writes to a device when all three are true:
- the EMS is in
SELF_TEST, simulator_urlis configured, and- the
device_type:fieldpair is in the allow-list.
Clearing any one of them disables device writes, because the settings value is
authoritative once the key exists. A run that needs a device write is refused
with 409 self_test_mode_required in any other mode; a run that touches no
device (self-test alarms only) is allowed in every mode.
5. Pitfalls
- Unregistered mode is silently replaced by IDLE. There is no error. Always do Step 1 first and verify with Step 8.5.
- Empty inputs are skipped. If a new field must be clearable, add
data-json-empty. - Blade views are compiled. After editing the blade, run
view:clearor the old markup keeps being served. - Missing
transkeys render as the raw key (e.g.cruds.emsSettings.Save_Self_Test). Add them in the same commit. - Do not change the seeded default mode. A plant that boots into a mode that suspends control is an outage.
data-json-pathmust match the JSON exactly, including the nesting. A typo creates a silent new branch in the config instead of an error.- The allow-list takes rule fields, not reading keys. Reading keys are the
prefixed ones (
gridMeter_frequency_hz); the alarm map rule field is not.
6. Connecting the mode to the ems-backend API
A mode with its own endpoints (SELF TEST added two: the diagnostic-test API and the alarm-diagnostics API) communicates in two directions, and each side has a fixed convention. Follow the existing implementations rather than inventing a third shape.
6.1 Backend: EnergySyc_EMS_ES100Cx12S/ems_settings_api.py
The EMS API is a plain BaseHTTPRequestHandler with hand-written routing in
do_GET / do_POST. One endpoint group touches five places:
- Import the service at the top of the module.
- Declare the base path as a constant, next to the others:
DEFAULT_ALARM_DIAGNOSTICS_PATH = "/api/v1/ems/diagnostics" - Add handler attributes, next to
diagnostic_path/diagnostic_service:alarm_diagnostics_path: str = DEFAULT_ALARM_DIAGNOSTICS_PATH alarm_diagnostics_service: Optional[AlarmDiagnosticService] = None - Claim the path in the list of paths the server recognises, and keep it
out of the readings fallback that would otherwise treat it as a device id:
and not path.startswith(self.alarm_diagnostics_path.rstrip("/") + "/") - Branch early: in
do_GETasif self._alarm_diagnostics_get(path): return, and indo_POSTbehind anif not path.startswith(base + "/"): returnguard, then route on the suffix (alarms/run,alarms/run-all,alarms/status/<id>,alarms/stop/<id>,reports/...).
Authentication is a bearer token, and it is only enforced when the server was
started with one (accept the Bearer prefix case-insensitively). A backend with
no token configured accepts the call without a header — which is why the
frontend may send none.
House convention for agent edits: wrap every added block in
# >>> DEEP-COPILOT-PATCH [TAG] (agent edit - see DEEPCOPILOT_PATCHES.md) and
# <<< DEEP-COPILOT-PATCH [TAG], then list it in DEEPCOPILOT_PATCHES.md. The
alarm-diagnostics routes carry the tag EMS-ALARM-DIAGNOSTICS.
Port note: the settings API listens on 8080 inside the compose network and is not published to the host. 1540 is the Modbus server, not the API, so a curl from the host to 1540 hangs instead of answering.
6.2 Frontend: reaching that API from a controller
config/ems.php holds the three keys: backend_url (default
http://ems-backend:8080), settings_api_token, and backend_token.
The conventions, as implemented in app/Http/Controllers/EmsSelfTestController.php:
- Build the URL from config, never hardcode it:
rtrim(config('ems.backend_url', 'http://ems-backend:8080'), '/'). The compose service nameems-backendresolves from insideems-frontend. - Send the token only when one is configured: try
ems.settings_api_token, fall back toems.backend_token, and add noAuthorizationheader when both are empty. - Always set a timeout — 15 s for reads, 30 s for queueing a run. An untimed
Httpcall to a restarting backend hangs the page instead of failing. - Return one failure shape for both callers: JSON with an
errorkey when$request->expectsJson(), otherwiseback()->with('error', ...). Keep it in a single helper so a form post and afetch()report the same detail. - Name every route in
routes/web.php(the SELF TEST ones areemsSelfTest,emsSelfTest.tests,emsSelfTest.run,emsSelfTest.status,emsSelfTest.report) and resolve them in Blade withroute(...), so the browser never carries a hardcoded path.
When a page and a tab must show the same data, give both the same JSON source (as the SELF TEST runner partial does) instead of duplicating the list in Blade.
6.3 Two traps that cost real debugging time
- New routes stay invisible until the route cache is cleared.
bootstrap/cache/routes-v7.phpis authoritative, so a plain pull is not enough: runphp artisan route:clear(aswww-data, see Step 8). The symptom otherwise is aroute()that works in Blade but a URL that 404s. - Blade's
@phpdirective must be used as a block. The compiler extracts@php ... @endphpwith a non-greedy match starting at the first@php, so an inline@php($x = [...])swallows every following line into a raw PHP block and the view stops compiling: the compiled file ends mid-markup and the page 500s. Related:@json([...])does not accept a multi-line array literal. Build the array in@php ... @endphpand pass the variable to@json($var).