In recent years, the demand for remote psychological support has been increasing, and online counseling has emerged as an important setting for clinical practice. Unlike face-to-face sessions, however, the assessment of nonverbal behavior is often more challenging in online environments. In particular, methods for quantitatively describing and analyzing the interactive dynamics of facial movements between therapist and client have not been sufficiently established. The present study develops a novel analytic framework designed to capture these dyadic facial actions during online counseling sessions.
Specifically, video recordings of therapist-client interactions were processed using MediaPipe (Google) to extract facial action units (AUs) frame by frame. The resulting multivariate time-series data of facial actions were subjected to non-negative matrix factorization (NMF), which reduced the complex facial movement patterns into a small number of interpretable components. NMF was chosen because its non-negativity constraint provides parts-based, additive representations that are particularly well suited for decomposing facial movement data into components that retain interpretability in a psychological and behavioral context. The optimal number of factors was determined based on reconstruction error and stability indices.
Subsequently, the temporal dynamics of the component scores were analyzed using vector autoregressive (VAR) modeling, Granger causality testing, and orthogonalized impulse response analysis. These methods enabled the identification of interdependencies between therapist and client facial actions, such as whether a therapist's facial movements systematically precede and predict the client's responses, or conversely, whether the client's movements drive changes in the therapist's behavior.
The proposed framework has the potential to deepen our understanding of nonverbal interaction in online counseling and to contribute to both clinical practice and professional training. In addition, future work may extend the method toward real-time automated feedback systems to support therapists in online environments.