Troubleshooting
Settings are not updating
Check the poller log first:
tail -n 50 /app/logs/ems_settings_poll.log
| Message | Cause | Fix |
|---|---|---|
settings refresh failed |
ems-frontend unreachable, DNS, or timeout |
Check both containers share a Docker network and url in ems_api_config.json |
HTTP 401 |
Wrong or expired bearer_token |
Update the token in ems_api_config.json |
pulled settings missing ems_operation_mode |
Payload shape changed | Inspect ems_pull_setting/ems_pull_setting.json |
Confirm the pull endpoint is reachable from inside ems-backend:
curl -s -o /dev/null -w '%{http_code}\n' \
-H 'Accept: application/json' \
-H "Authorization: Bearer $EMS_PULL_TOKEN" \
http://ems-frontend/api/v1/bess/GTW-ems100CX/ems-settings/get-ems
Pulled settings are rejected
The guard requires ems_operation_mode and a non-empty general_settings.pcs. Inspect what the portal actually returned:
python3 -c "import json;d=json.load(open('ems_pull_setting/ems_pull_setting.json'));print(sorted(d))"
If the settings moved to a new wrapper key, add that key to _unwrap_settings() in ems_settings_api.py.
The active cache is intentionally left unchanged while the guard fails, so the EMS keeps running on the last good configuration.
EMS is idle and not commanding the PCS
Check the active settings:
python3 -c "import json;d=json.load(open('emsConfig/ems/ems_control_setting.json'));print(d.get('ems_operation_mode'), len(d.get('general_settings',{}).get('pcs',{})))"
- An empty or invalid
ems_operation_modefalls back toIDLE. - Only PCS entries with
"status": "active"are dispatched. STOP_ALLandFFRare defined but not dispatched byems_main.py.- In
FREQUENCY_REGULATION, the slow loop is skipped and a separate fast process drives the PCS.
Frontend changes get reverted
The pull runs every 5 seconds and overwrites the active cache. If the pull endpoint still returns the previous value, the frontend change is replaced on the next pull. Persist the change on the frontend side, or set enabled to false in ems_api_config.json while testing locally.
Readings are empty
The API only returns files present in lastValue/. Confirm the collectors are running and the files are recent:
ls -l lastValue/
tail -n 20 /app/logs/pcs_client.log
HTTP API returns 401
EMS_SETTINGS_API_TOKEN in ems-backend and EMS_BACKEND_TOKEN in ems-frontend must match exactly, with no trailing whitespace. Restart both containers after changing them.
Laravel returns 502
Laravel cannot reach the backend. Confirm both containers are on the same Docker network and the service name is exactly ems-backend, then test from inside the Laravel container:
curl -v http://ems-backend:8080/api/v1/ems/health