device_duplicate.py — device folder manager
Reference for the script that keeps every device family's shared files in step
with its per-unit folders, creates folders for new devices, and reports the
state of the whole emsConfig device tree.
- Script:
EnergySyc_EMS_ES100Cx12S/device_duplicate.py - Runs against:
emsConfig/(default: theemsConfigfolder beside the script) - Writes to:
emsConfig/<family>/<prefix><n>/and, on--scaffold, new folders - Never touches: per-device files,
emsConfig/ems/ems_register_full.json,backup/folders
1. The problem it solves
Every device family in emsConfig stores its files in the same two-part shape:
emsConfig/<family_dir>/
<prefix>_main/ files shared by every unit of the family
<prefix>1/ one folder per unit
<prefix>2/
...
<prefix>10/
backup/ retired files, never read
The shared half is the alarm codes and sequences/. The per-device half
is the register map (and its model variants), the modbus configs and the
lastValue template — those carry each unit's own identity and addresses.
That split creates three chores that are easy to get wrong by hand:
- When an alarm code changes, every unit folder needs the same new file.
- When a device is added to
ems_control_setting.json, its folder has to be created and seeded before the build will accept it. - When a family drifts — one unit folder holding a stale alarm code — nothing notices until the build or the runtime misbehaves.
device_duplicate.py does all three, and can report on them without writing
anything.
2. Two names per family
A family has a directory and a folder prefix, and they are not always the
same word. This is why FAMILIES is a mapping rather than a list:
--family |
directory under emsConfig/ |
folder prefix | example folder |
|---|---|---|---|
pcs |
pcs |
pcs |
pcs/pcs1/ |
bmu |
bmu |
bmu |
bmu/bmu1/ |
grid |
grid_energy_meter |
grid_meter |
grid_energy_meter/grid_meter1/ |
solar |
solar_energy_meter |
solar_meter |
solar_energy_meter/solar_meter1/ |
ffr |
ffr_energy_meter |
ffrMeter |
ffr_energy_meter/ffrMeter1/ |
ups |
ups |
ups |
ups/ups1/ |
door |
door_sensor |
door_sensor |
door_sensor/door_sensor1/ |
temperature |
temperature_sensor |
temperature_sensor |
temperature_sensor/temperature_sensor1/ |
flood |
flood_sensor |
flood_sensor |
flood_sensor/flood_sensor1/ |
smoke |
smoke_sensor |
smoke_sensor |
smoke_sensor/smoke_sensor1/ |
heat |
heat_sensor |
heat_sensor |
heat_sensor/heat_sensor1/ |
sensorB |
sensor_block |
sensorB |
sensor_block/sensorB1/ |
ac |
air_condition_unit |
ac |
air_condition_unit/ac1/ |
<prefix>_main/ is the shared master for that family. It is not counted as a
device folder, even though it matches the <prefix>… naming.
3. When to use it
| Situation | Command |
|---|---|
You changed an alarm code in <prefix>_main/ and want it everywhere |
--sync (the default) |
| You want to know whether anything has drifted, without changing files | --check |
| You are about to sync and want to see the plan first | --dry-run |
You added a device to ems_control_setting.json |
--scaffold |
You created an empty <prefix>11/ by hand and want it seeded |
--scaffold |
| Something looks wrong and you need the whole resolved picture | --debug |
| You only care about one family | add --family <name> to any of the above |
Use it when
- The shared half of a family changed (alarm codes,
sequences/). - A device was added to or removed from the settings file.
- You are diagnosing "which unit folder is out of step".
- Before a build, as a cheap consistency gate —
--checkexits non-zero on drift.
Do not expect it to
- Copy per-device files. Register maps, modbus configs and
lastValuetemplates legitimately differ per unit; the script will not move them, and--scaffolddeliberately creates a folder without them so you can supply the real ones. - Delete anything. There is no prune or remove action. Removing a device is
a manual move into
backup/. - Know a new family. Adding a whole new family means editing
FAMILIES(and usuallySETTINGS_GROUPS) in the script.--scaffoldhandles new devices inside a known family. - Touch the merged map.
ems_register_full.jsonis produced by the two other scripts, not this one.
4. Command reference
python3 device_duplicate.py [action] [options]
Actions
Exactly one action is expected. If none is given, --sync runs.
| Flag | Writes? | What it does |
|---|---|---|
--sync |
yes | Default. Copies every shared file from <prefix>_main/ into each <prefix><n>/ that differs or is missing it. |
--check |
no | Reports drift per family and per folder. Exits 1 if any drift was found, 0 if everything matches. |
--dry-run |
no | Prints what --sync would copy, without copying. |
--scaffold |
yes | Creates folders for devices declared in the settings file that have no folder, and fills device folders that exist but are empty. Prints the per-device files you must supply. |
--debug |
no | Dumps the resolved state of every family and exits 0. |
--help |
no | Full usage text. |
--check and --dry-run overlap. If both are given, the script says so and
behaves as --check.
Options
| Flag | Default | Meaning |
|---|---|---|
--config <dir> |
<script dir>/emsConfig |
The emsConfig directory to operate on. |
--settings <rel> |
ems/ems_control_setting.json |
Settings file, relative to --config. Only used by --scaffold and --debug. |
--family <name> |
all |
Restrict to one family. Accepts any key from the table in §2, plus all. |
Exit codes
| Code | Meaning |
|---|---|
0 |
Success. With --check: no drift. |
1 |
--check only: drift was found. |
2 |
The --config directory does not exist. |
5. Where the device list comes from
--scaffold and --debug read ems_control_setting.json and compare its
declared units against the folders on disk. Each settings group maps to a
family through:
SETTINGS_GROUPS = { group: (unit-key stem, family dir, folder prefix) }
The stem is the word the settings file uses for a unit; the prefix is what the folder is called. They differ whenever names disagree:
| Settings group | unit key | folder it maps to |
|---|---|---|
pcs |
pcs1 |
pcs/pcs1/ |
bmu_units |
bmu1 |
bmu/bmu1/ |
grid_meters |
grid_meter1 |
grid_energy_meter/grid_meter1/ |
solar_meters |
solar_meter1 |
solar_energy_meter/solar_meter1/ |
ffr_meters |
ffrEnergyMeter1 |
ffr_energy_meter/ffrMeter1/ |
ups_units |
ups1 |
ups/ups1/ |
door_sensors |
door_sensor1 |
door_sensor/door_sensor1/ |
temperature_sensors |
temperature_sensor1 |
temperature_sensor/temperature_sensor1/ |
flood_sensors |
flood_sensor1 |
flood_sensor/flood_sensor1/ |
smoke_detectors |
smoke_detector1 |
smoke_sensor/smoke_sensor1/ |
heat_sensors |
heat_sensor1 |
heat_sensor/heat_sensor1/ |
sensor_blocks |
sensor_block1 |
sensor_block/sensorB1/ |
ac_units |
ac1 |
air_condition_unit/ac1/ |
Note the two rows where the settings key and the folder name genuinely disagree:
smoke_detector1 lives in smoke_sensor1/, and sensor_block1 lives in
sensorB1/. This is why the stem is tracked separately rather than derived.
Groups absent from SETTINGS_GROUPS are reported by --debug but never
scaffolded, because their layout has not been confirmed. Currently reported
as uncovered: bess_energy_meters, cooling_unit, frequency_meters, fss.
Adding one is a deliberate act: confirm the directory and prefix, add the row,
then --scaffold. Guessing would create wrongly-named folders.
6. Worked examples
Day-to-day: an alarm code changed
$ python3 device_duplicate.py --check
=== bmu : bmu/bmu_main -> 10 device folder(s) ===
bmu_alarm_code.json
bmu_alarm_code___bmu100.json
...
already matching: 7 folder(s)
out of step: 3 folder(s)
bmu/bmu3 bmu_alarm_code.json differs
bmu/bmu4 bmu_alarm_code.json differs
bmu/bmu8 bmu_alarm_code.json missing
(--check: nothing written)
...
DRIFT DETECTED - run without --check to refresh the folders
$ echo $?
1
Then apply it:
$ python3 device_duplicate.py
One family only
$ python3 device_duplicate.py --family bmu
A device was added to the settings file
Add bmu11 to general_settings.bmu_units, then:
$ python3 device_duplicate.py --scaffold --family bmu
scaffold: creating folders for devices declared in ems/ems_control_setting.json
=== bmu : bmu/ ===
NEW bmu11 (from bmu_units.bmu11 device_id=Cl1_P11)
+ bmu_alarm_code.json
+ bmu_alarm_code___bmu100.json
+ sequences/
copied 4 shared file(s) from bmu_main/
>>> NOW PUT THESE JSON FILES INSIDE bmu/bmu11/ <<<
bmu_modbus_config.json
bmu_register_full.json
bmu_register_full___bmu100.json
bmu_register_full___bmu200.json
bmu_register_full___bmu300.json
The folder is created and seeded with the shared half. The files listed under
the >>> NOW PUT THESE JSON FILES INSIDE … <<< banner are the per-device
ones — the script cannot invent them, and the build will be incomplete until
they exist.
An empty folder was created by hand
Creating bmu/bmu11/ yourself and running --scaffold gives:
=== bmu : bmu/ ===
EMPTY bmu/bmu11 (exists but holds nothing)
+ bmu_alarm_code.json
...
copied 4 shared file(s) from bmu_main/
>>> NOW PUT THESE JSON FILES INSIDE bmu/bmu11/ <<<
...
An empty folder is one holding no files anywhere underneath — a folder
containing only an empty sequences/ still counts as empty. Running
--scaffold twice is harmless: the second run reports that every declared
device already has a folder.
Diagnosing the whole tree
$ python3 device_duplicate.py --debug
config: /…/emsConfig
resolved config: /…/emsConfig
settings file: /…/emsConfig/ems/ems_control_setting.json
families: 13
---- ups
dir ups/ups
shared files 4 ups_alarm_code.json, ups_alarm_code___ups_10k.json, …
shared dirs sequences
device folders 1 ups1
declared 1 ups1
per-device ups_modbus_config.json, ups_register_full.json, …
...
settings groups with no scaffold mapping (4): bess_energy_meters, cooling_unit, frequency_meters, fss
--debug is the fastest way to answer "what does the script think the layout
is" without changing anything.
7. Reading the output
| Label | Meaning |
|---|---|
already matching: N folder(s) |
Units whose shared files are byte-identical to <prefix>_main/. |
out of step: N folder(s) |
Units with a shared file missing or differing. |
EMPTY <path> |
A device folder with no files anywhere underneath; --scaffold seeds it. |
SKIP: <path>_main does not exist |
The family has no shared master, so there is nothing to sync. If this is unexpected, the family's folder was renamed or never created. |
SKIP: <prefix>_main/ has no shared files |
The master exists but is empty. |
MISSING (in --debug) |
Declared in the settings file with no folder on disk. Run --scaffold. |
not declared (in --debug) |
A folder on disk with no corresponding settings entry. The build will not see it. |
>>> NOW PUT THESE JSON FILES INSIDE … <<< |
Per-device files you must supply by hand. |
Comparison is by SHA-256, so a whitespace-only change counts as drift. Files
matching the history pattern — .bak, .back, .backup, backupNN,
_backup, _clean_, write_queue, .orig, ~ — are ignored entirely: never
synced, never reported.
8. Relationship to the other scripts
Three scripts share this tree, and they do different jobs:
| Script | Direction | Writes |
|---|---|---|
generate_ems_register_full.py |
device folders → device maps | ems/ems_register_full.json |
ems_merge_register_map.py |
device maps + settings → EMS map | ems/ems_register_full.json, and annotates ems/ems_control_setting.json |
device_duplicate.py |
<prefix>_main/ → <prefix><n>/ |
device folders only |
device_duplicate.py is the one that never touches the settings file or the
merged map. Run it before the other two when the shared files changed, so
the build sees a consistent tree.
Two warnings carried over from the build scripts:
ems_merge_register_map.pyregenerates the wholeregister_mapblock of every settings unit it can map, includingems_base_address/ems_end_address/register_count. Those are EMS-side allocations in the 30xxx space, computed by that script — not device-side spans. Do not hand-write them.- A settings group that maps through
GROUP_TO_BLOCKbut has no matching entry in that script'sBLOCKStable is silently skipped during annotation. The symptom is aregister_mapthat keeps whatever was last written by hand.
9. Conventions for this script
Every Python script in this project must provide both --help and
--debug.
--help— not the bare argparse default: a real docstring describing the layout and behaviour, with an epilog of usage examples.--debug— dumps resolved state (paths, whether settings loaded, per-family counts, declared vs on-disk, missing and empty items) and exits without writing.
All three scripts implement both — device_duplicate.py,
generate_ems_register_full.py and ems_merge_register_map.py. Run
<script> --debug before a build to see the resolved state without writing.
Two more invariants worth preserving:
- Shared files are derived from
<prefix>_main/, never hardcoded. Adding a file to a master teaches the script about it automatically. - The per-device list shown by
--scaffoldis read off sibling folders, not hardcoded, so a family that grows a new per-device file type is reported correctly with no code change.
10. Troubleshooting
A family reports SKIP: … does not exist.
The directory or its <prefix>_main/ is missing or misnamed. Check the family
row in §2 against what is on disk; a renamed directory needs a FAMILIES edit.
Everything is "out of step" after a legitimate per-device change.
Per-device files are not compared, so this means a shared file differs in
those folders too. Check whether you edited a file inside a device folder that
should have been edited in <prefix>_main/.
--scaffold says a device is NEW but the folder exists.
The folder name does not match the expected <prefix><n>. --scaffold matches
strictly on the prefix plus digits; a folder named bmu_11 will not be seen as
bmu11.
--scaffold refuses to act.
It prints cannot scaffold <name> -- <prefix>_main/ does not exist, so the
layout is unknown. Create the master folder first, then re-run.
A new folder was created but the build ignores it.
The build's own tables are separate. A brand-new device needs its block
declared in generate_ems_register_full.py (and usually
ems_merge_register_map.py) as well as a folder.
Unsure what the script sees?
--debug first, always. It resolves and reports without writing.