Skip to content

Adding an EMS Operation Mode and its Configuration Settings

This guide is the checklist for adding a new EMS operational mode together with its settings page. It is written from the SELF_TEST mode, which was added in exactly this way, so every step below has a working reference implementation you can copy.

Read operation-modes.md first for what an operation mode is and how the control loop dispatches it.

1. What adding a mode actually touches

A mode is not one file. It spans four layers, and a mode that is registered in only some of them fails in a specific, confusing way:

Layer File If you forget it
Mode registry ems_operation_modes.py The mode is rejected and silently replaced by IDLE when it is written to the settings, so the EMS appears to accept the mode and then runs the plant as IDLE.
Dispatcher ems_main.py The loop falls through to a no-op or the previous branch; the mode runs but does nothing.
Settings structure ems_control_setting.template.json (tracked) and the runtime emsControl/ems/ems_control_setting.json The parameters the mode reads do not exist, so the mode uses hardcoded defaults.
Settings page emsConfigurationOperation.blade.php, cruds.php, EmsSettingTableSeeder.php An operator cannot see or change the parameters from the UI.

The mode string is the join between all four layers. Pick it once (SCREAMING_SNAKE_CASE, e.g. SELF_TEST), keep the function block name in PascalCase (Self_Test), and use the kebab-case form (self-test) for the tab id.

2. How the settings page works

Before editing the page it helps to know that almost nothing about it is per-mode. public/js/bess-config-mapper.js drives every tab generically:

  • A tab is three pieces of markup, with no JavaScript to write:
  • <li class="tab" data-tab="<tab-id>"> in the <ul class="tabs"> strip.
  • <div class="tab-pane" id="<tab-id>"> holding the inputs.
  • <button class="save-tab-btn" data-tab="<tab-id>"> inside a <div class="tab-save-section">.
  • Inputs bind to the settings JSON with data-json-path. On load the mapper reads each path out of the config and fills the input; on Save it walks every [data-json-path] inside that tab pane and writes the values back. The path is the literal JSON path, e.g. functions.Self_Test.alarm_diagnostic_test.appear_timeout_s.
  • Empty inputs are skipped by default. getValueFromInput() returns null for an empty box and the mapper then leaves the old value in place, so a field cannot be cleared. For a field that must be clearable, add data-json-empty to say what empty means, e.g. data-json-empty="" for a list of strings. This attribute is why the SELF_TEST allow-list can be emptied from the UI.
  • Checkboxes are switches: <label class="switch"> with an <input type="checkbox">; the value is read from checked.
  • Number inputs are type="number"; an empty box means "keep the old value", so always ship a sane default in the seeder.
  • Tabs can be permission gated with @can('Some_Permission') around the <li>. The Diagnostic tab is ungated; most others are gated.

The configuration round trip is:

ems_settings.scada_settings_json (DB)
        │  page load
        ▼
this.config in bess-config-mapper.js ──► inputs
        ▲                                   │  Save (save-tab-btn)
        │  these paths only                 ▼
        └────────────────────────── this.config updated in place
                                            │  main Save button
                                            ▼
                          ems_settings row  ──►  EMS pulls it
                                            ▼
                    emsControl/ems/ems_control_setting.json

3. Step-by-step

Step 1 — Register the mode (ems_operation_modes.py)

EMS_OP_SELF_TEST = "SELF_TEST"
...
ALL_EMS_OPERATION_MODES = (
    EMS_OP_IDLE,
    ...
    EMS_OP_SELF_TEST,   # add it to the tuple
)

Registering it is what makes the value survive validation. Skip this and the mode string is discarded in favour of IDLE.

Step 2 — Dispatch it (ems_main.py)

Add a branch in the control loop. If the mode must not run plant control, this is where you guarantee that:

    elif mode == ems_operation_modes.EMS_OP_SELF_TEST:
        # SELF TEST runs no plant control: the alarm diagnostic test drives
        # the registers, and a concurrent dispatch would fight it.
        pass

Step 3 — Add the settings structure

Add the function block to the tracked template, which is the versioned definition of the settings shape:

EnergySyc_EMS_ES100Cx12S/emsConfig/ems/ems_control_setting.template.json

  "functions": {
    "Self_Test": {
      "enabled": true,
      "description": "Reserves the EMS for the full alarm diagnostic test. ...",
      "alarm_diagnostic_test": {
        "include_self_test_alarms": true,
        "appear_timeout_s": 30,
        "clear_timeout_s": 45,
        "row_settle_s": 8,
        "max_alarms_per_device": 0,
        "simulator_url": null,
        "allow_simulator_register_writes": []
      }
    },

Note the runtime file emsControl/ems/ems_control_setting.json is gitignored on purpose - its comment says the key structure is versioned in ems_control_setting.template.json instead. Only edit the runtime file on a live host, and never commit it.

Step 4 — Add the tab to the settings page

In resources/views/emsConfigurationOperation.blade.php, add the three pieces of markup. Keep the tab next to the related ones:

{ 1. the tab button, inside <ul class="tabs"> }
<li class="tab" data-tab="self-test">
    <i class="fas fa-vial"></i>
    <span>{{ trans('cruds.emsSettings.Self_Test') }}</span>
</li>

{ 2. the pane, inside <div class="tab-content"> }
<div class="tab-pane" id="self-test">
    <h5 class="tab-title">{{ trans('cruds.emsSettings.Self_Test_Settings') }} </h5>
    <div class="settings-grid">
        <div class="row">
            <div class="col-md-6">
                <div class="card">
                    <h6><i class="fas fa-vial"></i>{{ trans('cruds.emsSettings.Self_Test_Mode') }} </h6>
                    <div class="form-group">
                        <label>{{ trans('cruds.emsSettings.Appear_Timeout') }} </label>
                        <input type="number" step="1" min="1"
                            data-json-path="functions.Self_Test.alarm_diagnostic_test.appear_timeout_s">
                        <span class="field-info">{{ trans('cruds.emsSettings.Appear_Timeout_Info') }} </span>
                    </div>
                </div>
            </div>
        </div>
    </div>

    { 3. the save button }
    <div class="tab-save-section">
        <button class="btn save-tab-btn" data-tab="self-test" id="saveSelfTestSettings">
            <i class="fas fa-save"></i> {{ trans('cruds.emsSettings.Save_Self_Test') }}
        </button>
    </div>
</div>

A clearable field carries data-json-empty:

<textarea rows="3" data-json-empty=""
    data-json-path="functions.Self_Test.alarm_diagnostic_test.allow_simulator_register_writes"></textarea>

Step 5 — Add the labels (resources/lang/en/cruds.php)

Every trans('cruds.emsSettings.X') you used needs a key in the emsSettings array, otherwise the page shows the raw key:

            'Self_Test' => 'Self Test',
            'Self_Test_Settings' => 'Self Test Settings',
            'Save_Self_Test' => 'Save Self Test Settings',

Step 6 — Add the block to the seeder (database/seeders/EmsSettingTableSeeder.php)

The seeder holds the whole settings document as scada_settings_json. Add the same function block there.

Keep the already selected mode untouched:

            'ems_operation_mode' => 'MANUAL',
            'ui_state' => [ 'radio_selected_function' => 'MANUAL', ... ],

A plant must never boot into a mode that suspends control. The new mode is selectable, not selected.

Step 7 — Optional: add the tab to the debug map

public/js/bess-config-mapper.js, getTabConfig() only feeds the console log; add it for consistency:

            'self-test': this.config.functions.Self_Test,

Step 8 — Deploy and verify

# Always run artisan as the web user. A root run in this container leaves
# root-owned files under storage/ and bootstrap/cache, which php-fpm (www-data)
# can read but not overwrite: every later view recompile then fails with
# "file_put_contents(...): Permission denied" and returns 500 to real users.
#
# route:clear is what makes a NEW route visible at all, because
# bootstrap/cache/routes-v7.php is authoritative at runtime.
docker exec -u www-data ems-frontend php artisan route:clear
docker exec -u www-data ems-frontend php artisan view:clear
docker exec -u www-data ems-frontend php artisan config:clear

Then confirm, in this order:

  1. Renders - the tab appears and shows the values, not the raw trans keys.
  2. Saves - change a value, Save, and reload; the value persists.
  3. Persists - the DB row carries it: SELECT JSON_EXTRACT(scada_settings_json, '$.functions.Self_Test') FROM ems_settings;
  4. Reaches the EMS - after the settings pull, the live file matches: docker exec ems-backend python3 -c "import json;print(json.load(open('/app/EnergySyc_EMS_ES100Cx12S/emsControl/ems/ems_control_setting.json'))['functions']['Self_Test'])"
  5. Mode is accepted - select it and confirm the EMS reports it back rather than falling back to IDLE. If it falls back, Step 1 is missing.
  6. Mode does nothing harmful - in the new mode the plant must be idle.

4. SELF_TEST settings reference

functions.Self_Test.enabled — shows SELF TEST in the Operational Functions selector in the Operation tab.

functions.Self_Test.alarm_diagnostic_test.include_self_test_alarms — also exercise the SELFTEST-nnn alarms, which are injected without touching any device.

functions.Self_Test.alarm_diagnostic_test.max_alarms_per_device — 0 tests every configured alarm of the device; any other value caps the run.

functions.Self_Test.alarm_diagnostic_test.appear_timeout_s / clear_timeout_s — how long to wait for the triggered alarm to appear, and for it to disappear after the trigger is reverted, before recording a timeout.

functions.Self_Test.alarm_diagnostic_test.row_settle_s — delay before reading active_alarm_records, so the 1 Hz collector has written its row.

functions.Self_Test.alarm_diagnostic_test.simulator_url — base URL of the simulator register API, e.g. http://bess-simulator-backend:8000. Empty means no device register is ever written.

functions.Self_Test.alarm_diagnostic_test.allow_simulator_register_writes — the device_type:field pairs the test may write, e.g. grid_energy_meter:frequency_hz. Accepts a JSON list or the comma separated string the settings page writes. The second element is the rule field from the device alarm map (frequency_hz), not the reading key (gridMeter_frequency_hz).

Safety: three independent opt-ins

A diagnostic test only writes to a device when all three are true:

  1. the EMS is in SELF_TEST,
  2. simulator_url is configured, and
  3. the device_type:field pair is in the allow-list.

Clearing any one of them disables device writes, because the settings value is authoritative once the key exists. A run that needs a device write is refused with 409 self_test_mode_required in any other mode; a run that touches no device (self-test alarms only) is allowed in every mode.

5. Pitfalls

  • Unregistered mode is silently replaced by IDLE. There is no error. Always do Step 1 first and verify with Step 8.5.
  • Empty inputs are skipped. If a new field must be clearable, add data-json-empty.
  • Blade views are compiled. After editing the blade, run view:clear or the old markup keeps being served.
  • Missing trans keys render as the raw key (e.g. cruds.emsSettings.Save_Self_Test). Add them in the same commit.
  • Do not change the seeded default mode. A plant that boots into a mode that suspends control is an outage.
  • data-json-path must match the JSON exactly, including the nesting. A typo creates a silent new branch in the config instead of an error.
  • The allow-list takes rule fields, not reading keys. Reading keys are the prefixed ones (gridMeter_frequency_hz); the alarm map rule field is not.

6. Connecting the mode to the ems-backend API

A mode with its own endpoints (SELF TEST added two: the diagnostic-test API and the alarm-diagnostics API) communicates in two directions, and each side has a fixed convention. Follow the existing implementations rather than inventing a third shape.

6.1 Backend: EnergySyc_EMS_ES100Cx12S/ems_settings_api.py

The EMS API is a plain BaseHTTPRequestHandler with hand-written routing in do_GET / do_POST. One endpoint group touches five places:

  1. Import the service at the top of the module.
  2. Declare the base path as a constant, next to the others:
    DEFAULT_ALARM_DIAGNOSTICS_PATH = "/api/v1/ems/diagnostics"
    
  3. Add handler attributes, next to diagnostic_path / diagnostic_service:
    alarm_diagnostics_path: str = DEFAULT_ALARM_DIAGNOSTICS_PATH
    alarm_diagnostics_service: Optional[AlarmDiagnosticService] = None
    
  4. Claim the path in the list of paths the server recognises, and keep it out of the readings fallback that would otherwise treat it as a device id:
    and not path.startswith(self.alarm_diagnostics_path.rstrip("/") + "/")
    
  5. Branch early: in do_GET as if self._alarm_diagnostics_get(path): return, and in do_POST behind an if not path.startswith(base + "/"): return guard, then route on the suffix (alarms/run, alarms/run-all, alarms/status/<id>, alarms/stop/<id>, reports/...).

Authentication is a bearer token, and it is only enforced when the server was started with one (accept the Bearer prefix case-insensitively). A backend with no token configured accepts the call without a header — which is why the frontend may send none.

House convention for agent edits: wrap every added block in # >>> DEEP-COPILOT-PATCH [TAG] (agent edit - see DEEPCOPILOT_PATCHES.md) and # <<< DEEP-COPILOT-PATCH [TAG], then list it in DEEPCOPILOT_PATCHES.md. The alarm-diagnostics routes carry the tag EMS-ALARM-DIAGNOSTICS.

Port note: the settings API listens on 8080 inside the compose network and is not published to the host. 1540 is the Modbus server, not the API, so a curl from the host to 1540 hangs instead of answering.

6.2 Frontend: reaching that API from a controller

config/ems.php holds the three keys: backend_url (default http://ems-backend:8080), settings_api_token, and backend_token.

The conventions, as implemented in app/Http/Controllers/EmsSelfTestController.php:

  • Build the URL from config, never hardcode it: rtrim(config('ems.backend_url', 'http://ems-backend:8080'), '/'). The compose service name ems-backend resolves from inside ems-frontend.
  • Send the token only when one is configured: try ems.settings_api_token, fall back to ems.backend_token, and add no Authorization header when both are empty.
  • Always set a timeout — 15 s for reads, 30 s for queueing a run. An untimed Http call to a restarting backend hangs the page instead of failing.
  • Return one failure shape for both callers: JSON with an error key when $request->expectsJson(), otherwise back()->with('error', ...). Keep it in a single helper so a form post and a fetch() report the same detail.
  • Name every route in routes/web.php (the SELF TEST ones are emsSelfTest, emsSelfTest.tests, emsSelfTest.run, emsSelfTest.status, emsSelfTest.report) and resolve them in Blade with route(...), so the browser never carries a hardcoded path.

When a page and a tab must show the same data, give both the same JSON source (as the SELF TEST runner partial does) instead of duplicating the list in Blade.

6.3 Two traps that cost real debugging time

  • New routes stay invisible until the route cache is cleared. bootstrap/cache/routes-v7.php is authoritative, so a plain pull is not enough: run php artisan route:clear (as www-data, see Step 8). The symptom otherwise is a route() that works in Blade but a URL that 404s.
  • Blade's @php directive must be used as a block. The compiler extracts @php ... @endphp with a non-greedy match starting at the first @php, so an inline @php($x = [...]) swallows every following line into a raw PHP block and the view stops compiling: the compiled file ends mid-markup and the page 500s. Related: @json([...]) does not accept a multi-line array literal. Build the array in @php ... @endphp and pass the variable to @json($var).