Abstract. VitalMotion is a dual-modal monitoring system for unsupervised home-based
rehabilitation. It concurrently estimates movement quality from monocular video and
physiological load from cardiac inter-beat intervals, then aligns both on a common
time base. The design premise is that neither modality is individually sufficient: a
kinematic score cannot distinguish limited capacity from fatigue, and a heart-rate reading
cannot attribute elevated load to a specific movement fault. Temporal co-registration of
the two streams enables attribution that neither supports alone.
I. Vision Layer — Kinematic Assessment
Body pose is recovered from a single RGB camera using a 33-landmark topology
(MediaPipe Pose, Lugaresi et al., 2019). No depth sensor is required: joint angles are
computed as the planar angle subtended at vertex b by landmark triplets
(a, b, c), and all length-derived quantities are normalised by inter-shoulder
distance, which approximately cancels the unknown camera-to-subject scale.
Landmark estimates exhibit frame-to-frame jitter that propagates non-linearly into angle
estimates. A 1€ adaptive low-pass filter (Casiez et al., CHI 2012) is applied to
each joint-angle sequence, imposing a motion-continuity prior: the cutoff frequency scales
with the estimated angular velocity, yielding strong smoothing at low speed (jitter
suppression) and weak smoothing at high speed (preserved responsiveness).
Per-repetition movement quality is defined not by terminal joint angle alone — a
target angle may be reached through compensation rather than through isolated limb
effort, a distinction central to post-stroke motor relearning. Quality is therefore
penalised by observed trunk recruitment:
Q = Speak · (1 − min(1, C / Cref) · λ)
where Speak is the maximal pose-conformance score attained within
the repetition, C the trunk-compensation ratio, Cref a
normalising reference (0.35), and λ the maximum penalty fraction (0.40).
Two kinematic descriptors reproduce the quantitative framework of
Wagh et al. (2025): (i) the trunk-compensation ratio, the displacement of the
trunk centroid — the mean of bilateral shoulder and hip landmarks — normalised
by shoulder width; and (ii) the bivariate variable error, quantifying endpoint
repeatability across repetitions:
BVE = √( Var(x) + Var(y) )
computed over movement endpoints expressed relative to the trunk centroid, so that whole-body
translation within the frame does not inflate the measure.
II. Physiological Layer — Autonomic Load
Photoplethysmographic (PPG) signals acquired at the fingertip are dominated by motion
artefact during active movement. Pulse peaks are localised via wavelet-domain weak-feature
extraction, exploiting the scale separation between the pulsatile component and
broadband motion noise, yielding a sequence of inter-beat (RR) intervals.
Rather than thresholding heart rate alone, the system quantifies the ordinal complexity
of the RR sequence via permutation entropy (Bandt & Pompe, 2002). For embedding
dimension m and delay τ, each sub-sequence is mapped to its
rank-order pattern π, and the normalised Shannon entropy of the pattern
distribution is:
H(m) = − Σπ p(π) ln p(π) / ln(m!)
with m = 3, τ = 1. The measure is ordinal and therefore robust
to monotonic amplitude drift. It is bounded on [0, 1]: a strictly monotonic series yields
H = 0, an i.i.d. series yields H → 1. Deviation from the individual's resting entropy
reflects altered autonomic modulation and can precede a frank heart-rate excursion.
Anomaly detection is individualised rather than population-referenced. The first 40
accepted beats establish subject-specific means and standard deviations for smoothed heart
rate and entropy; thereafter a sample is flagged when it exceeds
μ + kσ (k = 2.5 for rate, 2.0 for entropy) or an absolute safety ceiling.
RR series are pre-processed for artefact. A missed beat approximately doubles an interval and
a double-count approximately halves it, either of which corrupts the entropy estimate. Intervals
outside physiological bounds, or deviating >30% from the running median, are rejected —
but only when isolated. Three consecutive out-of-range beats are instead interpreted as a
genuine level shift (e.g. rapid onset of exertional tachycardia) and accepted, so that
artefact rejection cannot mask a true clinical excursion.
III. Fusion Layer — Temporal Co-registration
Both streams are resampled onto a common 2 Hz time base. Because the modalities update at
very different rates (pose ≈ 30 Hz, cardiac ≈ 1–2 Hz), instantaneous sampling
would alias high-rate jitter; values are instead accumulated within each window and averaged
before a first-order exponential smoother is applied for display.
A three-state machine segments each session into rest → load → recovery.
Critically, state transitions are driven by the vision layer — detected movement
activity — and never by the cardiac signal itself. This unidirectional coupling is what
distinguishes fusion from two co-displayed instruments: it establishes movement as the
stimulus and cardiac dynamics as the response, permitting causal reading of the pair.
Two derived indices summarise the cardiovascular response: ΔHR, the load-phase
peak relative to resting baseline; and heart-rate recovery (HRR), the decrement within
60 s of cessation. HRR is an established marker of autonomic reactivation, with
≤ 12 bpm at one minute reported as prognostically unfavourable
(Cole et al., NEJM, 1999) — the threshold adopted here for flagging, not diagnosis.
IV. Scope and Limitations
-
Not clinically validated. Kinematic descriptors have not been compared against
marker-based optical motion capture. This is a methodological reproduction and engineering
prototype, not a validated instrument, and outputs are not a diagnostic basis.
-
Monocular estimation. Pose is recovered in the image plane; movements with
substantial out-of-plane components are subject to projective foreshortening that
normalisation does not fully correct.
-
Sensor provenance. The system accepts input from a standard BLE heart-rate service,
from custom PPG hardware, or from a clearly-labelled synthetic generator used for
demonstration. Simulated data is never presented as measurement. Devices reporting only
integer heart rate, rather than beat-to-beat intervals, cannot support a faithful entropy
estimate.
-
Advisory generation. Post-session guidance is produced by a language model
constrained to training advice, with a deterministic rule-based fallback. It performs no
diagnosis and defers to a rehabilitation clinician whenever safety thresholds are exceeded.
References
- Bandt, C., & Pompe, B. (2002). Permutation entropy: a natural complexity measure for time series. Physical Review Letters, 88(17), 174102.
- Casiez, G., Roussel, N., & Vogel, D. (2012). 1€ filter: a simple speed-based low-pass filter for noisy input in interactive systems. Proc. CHI '12, 2527–2530.
- Cole, C. R., Blackstone, E. H., Pashkow, F. J., Snader, C. E., & Lauer, M. S. (1999). Heart-rate recovery immediately after exercise as a predictor of mortality. New England Journal of Medicine, 341(18), 1351–1357.
- Lugaresi, C., et al. (2019). MediaPipe: a framework for building perception pipelines. arXiv:1906.08172.
- Wagh, A., et al. (2025). Kinematic quantification of compensatory movement in post-stroke upper-limb rehabilitation. Journal of NeuroEngineering and Rehabilitation, 22, 268.