We use cookies to ensure our website works properly and to personalise your experience. Cookies policy
Department of Master of Computer Applications Progressive Education Society’s Modern College of Engineering, Pune, Maharashtra, India
Emotional distress often goes unnoticed, and text tools that turn what people write into emotion signals could support self-awareness, yet many such tools are English-only, give a single score, and say little about their errors, privacy or limits. This paper presents MindCare AI, an end-to-end prototype that predicts multi-label emotions (27 emotions plus neutral) from short text. A TF-IDF (1–2-gram) representation feeds 28 one-vs-rest logistic regressions trained on the official GoEmotions splits (43,410 / 5,426 / 5,427 comments), with a single decision threshold (0.45) chosen on validation data only. On the official test split the model reaches macro-F1 0.445, micro-F1 0.514, micro precision 0.430, micro recall 0.638, subset accuracy 0.276 and Hamming loss 0.0503; the top-scoring label is one of the true labels for 56.9% of comments. Per-label F1 ranges from 0.888 (gratitude) to 0.129 (relief), and eight of 28 labels stay below 0.30. The model over-predicts (1.73 labels per comment against 1.17 annotated) and assigns neutral to 31.6% of comments whose annotation is not neutral. The classifier is embedded in a Flask web application with registration, opt-in text storage, per-user history, a JSON API, an English / Hindi / Marathi interface with a small phrase-and-emoji normalisation lexicon, rule-based suggestions, a 10-item self-check questionnaire and a keyword-based crisis notice. These last four components have not been validated, and transformer baselines were not run. This is a single-seed study of a prototype; it is an awareness aid, not a diagnostic tool.
Mental-health conditions are common: the World Health Organization estimated that almost one billion people were living with a mental disorder in 2019 [1], and India’s National Mental Health Survey (2015–16) found that roughly one adult in ten had a current mental morbidity, with a wide gap between need and treatment [2]. People usually put their feelings into words, in messages, notes and posts, long before they meet a professional. Software that turns such text into emotion signals could help a person reflect on how they feel, but only if it is honest about what it can and cannot do.
Early emotion datasets and tools used a handful of basic emotions [3, 4]. GoEmotions [5] moved to 27 fine-grained emotions plus neutral, annotated on about 58,000 English Reddit comments, and allows several labels per comment. Fine-grained labels are hard to predict: many are close in meaning and annotators disagree, so even a fine-tuned BERT model is reported at a macro-F1 of only about 0.46 [5]. Practical tools add further problems: most work only in English, they seldom show the threshold or model version behind a prediction, and they rarely state clearly that an emotion label is not a diagnosis.
This paper documents a complete prototype, MindCare AI, and reports its evaluation on the official GoEmotions test split. Our contributions are modest and stated as such:
We do not claim state-of-the-art accuracy. The model is deliberately simple so that it is fast, inspectable and runs on a CPU, and transformer models were not evaluated (Section VI-G). Our official-split protocol matches that of the GoEmotions paper, but we did not re-run its models, so we make no head-to-head comparison.
II. PROBLEM STATEMENT AND OBJECTIVES
People find it hard to track their own emotional state over time, and services cannot read free text at scale. Automated emotion tools often (i) support English only, (ii) output one label or a polarity score, (iii) hide thresholds and model versions, (iv) keep user text by default, and (v) blur the line between emotional awareness and clinical diagnosis. The research therefore targets multi-label emotion prediction, transparent evaluation, privacy-conscious deployment, multilingual access and clear safety boundaries.
Objectives. The project has the following objectives, each described with the purpose it serves:
Objectives 1–5 are met at the implementation level. The held-out evaluation (Section IX) shows that objectives 1 and 3 are only partly achieved: the classifier recovers true labels often (recall 0.638) but with modest precision (0.430) and a macro-F1 of 0.445. The baselines and ablations of objective 6 have not yet been run.
III. LITERATURE SURVEY
A. Emotion taxonomies and datasets
Basic-emotion theories [3] and wheel models [4] underlie most emotion datasets. SemEval-2007 Task 14 [6] annotated news headlines, and CARER [7] labelled tweets with six emotions. GoEmotions [5] is larger and finer: 27 emotions plus neutral on Reddit comments, with multiple labels per comment, which is the dataset used here.
B. Bag-of-words and linear models
TF-IDF weighting [8] with a linear classifier is a classic, strong baseline for text. Lexicon methods such as the NRC word–emotion lexicon [9] and VADER [10] need no training but score words in isolation. Linear models over n-grams remain competitive for many text tasks and train quickly [11]. Multi-label problems are often handled by training one binary classifier per label (binary relevance, or one-vs-rest) [12], here with the liblinear solver [13].
C. Transformer models
Pre-trained transformers such as BERT [14], RoBERTa [15] and the smaller DistilBERT [16] are the usual route to higher accuracy on text classification. The GoEmotions authors fine-tuned BERT and reported a macro-F1 of about 0.46 on the test split [5]. These models need more memory and compute than a linear model, which matters for a small CPU-only deployment.
D. Mental-health tools and screening instruments
Systematic reviews of mental-health chatbots find promising but uncertain evidence and stress safety [17]; Woebot is a well-known example of a conversational agent evaluated in a trial [18]. Validated screening questionnaires such as PHQ-9 [19] and GAD-7 [20] are the clinical reference for depression and anxiety symptoms. Work on predicting mental-health status from social-media text warns that validity, bias and ethics are often under-examined [21]. These warnings are why this project presents emotion labels as awareness signals and not as diagnoses.
E. Indian languages and code-mixing
Multilingual encoders such as XLM-R [22] and MuRIL [23] cover Hindi and Marathi, benchmarks such as GLUECoS [24] address code-switched text, and L3CubeMahaSent [25] provides Marathi sentiment data. We did not use any of these; our Hindi and Marathi support is a small lexicon in front of an English-trained model (Section VII-B).
F. Comparison of the surveyed approaches
Table 1 compares the surveyed approaches with the proposed system.
Table 1: Comparison of surveyed approaches with the proposed system.
| Approach | Task | Relation to this work |
|---|---|---|
| Lexicons (NRC, VADER) [9, 10] | Word-level emotion or polarity scores | Simplest alternative; not evaluated here |
| TF-IDF + linear models [8, 11] | Text classification | Method family of this work |
| Fine-tuned BERT-type models [14, 16] | Fine-grained emotion on GoEmotions | Likely stronger, heavier; not evaluated here |
| MuRIL, XLM-R [22, 23] | Multilingual and Indic text | Possible replacement for the lexicon layer |
| Mental-health chatbots [17, 18] | Conversational support | Different goal; this work is not conversational |
| This work | GoEmotions, 28 labels, multi-label; web app | Single seed; wellbeing modules unvalidated |
IV. RESEARCH GAP
Fine-grained emotion classification on GoEmotions is not new, and this work does not claim a new task or model. It addresses several gaps that remain around such classifiers, each tackled here only at the system level.
The first gap is transparency of evaluation. Emotion tools are often described by one headline score. We use the official splits, choose the threshold on validation data only, report all 28 per-label results, show how sensitive the scores are to the threshold, and audit duplicate text across splits. The audit also corrected a figure in our own project notes (Section V-E).
The second gap is access and safety in an Indian setting. Most emotion tools take English input only. We add English, Hindi and Marathi interface text, a self-check questionnaire, suggestions and a crisis notice, with the openly stated caveat that the Hindi and Marathi handling is a small phrase lexicon and that none of these components has been validated.
The third gap is baselines and deployment evidence. Controlled comparisons against transformer models and per-label thresholds are specified in Section VI-G but have not been performed, and end-to-end latency has not been benchmarked, so the contribution of individual design choices remains unmeasured.
V. DATASET AND PREPROCESSING
A. GoEmotions
GoEmotions [5] contains English Reddit comments labelled with 27 emotions or neutral; a comment can carry several labels. We use the simplified release with its official train, dev and test files, each a tab-separated file of text, comma-separated label ids and comment id. The 28 labels are admiration, amusement, anger, annoyance, approval, caring, confusion, curiosity, desire, disappointment, disapproval, disgust, embarrassment, excitement, fear, gratitude, grief, joy, love, nervousness, optimism, pride, realization, relief, remorse, sadness, surprise and neutral. Each comment becomes a 28-dimensional multi-hot vector. Comments are short (mean 12.7 words in the test split). No Hindi or Marathi data are used for training or evaluation. The pipeline is shown in Fig. 1.

Figure 1: Data preparation pipeline for GoEmotions.
B. Text representation
No stemming or stop-word removal is applied. The scikit-learn vectoriser lower-cases the text and uses its default token pattern, which keeps words of two or more characters and ignores punctuation and emoji. Consequently an emoji or an unmatched Devanagari word contributes nothing to the TF-IDF vector (Section IX-D shows the effect).
C. Official partition
Table 2: Official GoEmotions partition used in this study.
| Split | Comments | Role |
|---|---|---|
| Train | 43,410 | Fit TF-IDF and classifiers |
| Validation (dev) | 5,426 | Select the threshold |
| Test | 5,427 | Report final results once |
Because the official splits are random samples of the same Reddit data, the test split measures performance on comments like the training comments. It does not measure performance on diary-style text, other platforms or other languages.
D. Class distribution
Labels are strongly imbalanced (Table 3). Neutral appears in about a third of the comments, whereas grief appears in 0.18% of training comments (77 of 43,410) and 6 test comments. Because a comment can have several labels, the percentages do not sum to 100. The three splits have very similar label profiles.
Table 3: Share of comments carrying each label (selected labels) and label cardinality.
| Label | Train (%) | Val. (%) | Test (%) |
|---|---|---|---|
| neutral | 32.76 | 32.55 | 32.93 |
| admiration | 9.51 | 8.99 | 9.29 |
| approval | 6.77 | 7.32 | 6.47 |
| annoyance | 5.69 | 5.58 | 5.90 |
| embarrassment | 0.70 | 0.65 | 0.68 |
| nervousness | 0.38 | 0.39 | 0.42 |
| pride | 0.26 | 0.28 | 0.29 |
| relief | 0.35 | 0.33 | 0.20 |
| grief | 0.18 | 0.24 | 0.11 |
| Labels per comment | 1.18 | 1.18 | 1.17 |
| Multi-label comments (%) | 16.4 | 16.2 | 15.4 |
E. Duplicate-text audit
We compared lower-cased, punctuation-stripped comment text between splits. There are 98 distinct strings shared by train and validation, 90 by train and test, and 25 by validation and test. Counted per comment, 109 of the 5,427 test comments (2.0%) and 113 of the 5,426 validation comments (2.1%) have text that also occurs in the training split. Our project notes had described this overlap as about 0.2% of test examples, which is inconsistent with these counts (90 shared strings against 5,427 test comments is already 1.7%); the per-comment figure is 2.0%. The official splits were not altered. To gauge the effect, we re-scored the test comments whose text does not occur in training (n = 5,318): macro-F1 is 0.4426 and micro-F1 0.5118, against 0.4451 and 0.5140 on the full split, so the overlap changes the scores only slightly. A model re-trained on de-duplicated data is future work.
VI. PROPOSED METHODOLOGY
A. TF-IDF representation
Each comment is mapped to a sparse vector of 1–2-gram features. With sublinear term frequency, the weight of term t in document d is
wt,d = (1 + ln tft,d) · idft, idft = ln ((1 + n)/(1 + dft)) + 1 (1)
and every vector is L2-normalised [8, 26]. Terms must occur in at least two training comments (min_df = 2); at most 200,000 features are kept, and 58,338 were retained. The vectoriser is fitted on training text only.
B. One-vs-rest logistic regression
For each label c a separate L2-regularised logistic regression gives a score
pc(x) = 1 / (1 + exp (−(wc·x + bc))) (2)
with C = 4.0, the liblinear solver and balanced class weights, so that within each binary problem errors on the rarer class are weighted more heavily. The model therefore has 28 × (58,338 + 1) = 1,633,492 parameters. Because of the class weights, the scores are not calibrated probabilities, and the application shows them as “model scores”.
C. Decision threshold
A label is predicted when its score reaches a single global threshold τ:
Å·c = 1 [pc(x) ≥ τ] (3)
τ was searched over 0.10, 0.15, …, 0.90 and chosen to maximise validation macro-F1, giving τ = 0.45. The test split played no part in this choice. The application also lists the top five scores for every comment, so a user still sees ranked evidence when no score reaches τ (this happens for 2.7% of test comments).
D. Evaluation metrics
Metrics are computed from the 28-column prediction matrix. For each label we report precision P = TP/(TP+FP), recall R = TP/(TP+FN) and F1 = 2PR/(P+R). Macro-F1 is the unweighted mean of the 28 per-label F1 values (neutral included) and is the headline figure because it weights rare labels equally; micro-F1 pools all decisions. Subset accuracy is the share of comments whose full label set is exactly right, Hamming loss is the share of wrong label decisions, and the top-1 hit rate is the share of comments whose highest-scoring label is one of the true labels.
E. Training configuration
Table 4: Training configuration.
| Item | Configuration |
|---|---|
| Libraries | scikit-learn 1.8.0, NumPy 2.4.4, pandas 3.0.2, joblib 1.5.3 |
| Vectoriser | TfidfVectorizer: 1–2-grams, min_df 2, sublinear tf, max 200,000 features (58,338 kept) |
| Classifier | OneVsRest(LogisticRegression): C 4.0, liblinear, class_weight balanced, max_iter 300 |
| Parameters | 28 × (58,338 + 1) = 1,633,492 |
| Threshold | Global τ, grid 0.10–0.90 (step 0.05), maximising validation macro-F1: τ = 0.45 |
| Tuning | Only τ is tuned in the training script; no search over C, n-grams or weights is documented |
| Seed | 42; single run, no confidence intervals |
| Artefact | Bundle (vectoriser, classifier, labels, τ, version) saved with joblib, 13.1 MB |
F. Comparability
Our protocol (official splits, 28 labels, macro-F1) is the one used by the GoEmotions authors [5], who report a macro-F1 of about 0.46 for fine-tuned BERT. We did not re-run their model, and details such as the thresholding rule may differ, so the two numbers should not be ranked against each other. An earlier configuration of ours (different hyper-parameters, τ = 0.55), as recorded in the project notes and not re-run for this paper, had test macro-F1 0.4544, micro-F1 0.5157 and subset accuracy 0.2898. Differences of about 0.01 between settings are of the same size as the validation–test gap (Section IX-C) and should not be over-interpreted.
G. Planned baselines and ablations
The most important next step is a stronger evaluation: several seeds or bootstrap intervals, and the comparisons in Table 5 under the same protocol. Only experiments actually performed will be reported.
Table 5: Planned baselines and ablations (not yet run).
| Configuration | Question answered |
|---|---|
| A. Proposed (TF-IDF 1–2-grams, balanced LR, global τ) | Reference |
| B. Same without class weights | Value of balanced weights |
| C. Unigrams only | Value of bigrams |
| D. Per-label thresholds tuned on validation | Value of label-specific τ |
| E. Linear SVM or Naive Bayes | Choice of linear learner |
| F. DistilBERT / RoBERTa fine-tuned on GoEmotions | Gap to transformers |
| G. Multilingual encoder on labelled Hindi / Marathi text | Accuracy of the lexicon layer |
| H. Training on de-duplicated data | Effect of duplicate text |
VII. WELLBEING AND SAFETY COMPONENTS
A. Transparent output
For every analysis the application shows the predicted labels, the top five scores, the threshold and the model version, and states that scores are model outputs and not probabilities of having a mental-health condition. We did not implement any attribution method. Because the classifier is linear, the n-grams that drive a prediction could be read from its weights; this is future work and is not claimed here.
B. Hindi, Marathi and emoji normalisation
Before classification, 36 Hindi, Marathi and emoji entries (for example words for “happy”, “sad”, “worried”, “thank you” and several emoji) are replaced by English words by plain substring replacement, and the English-trained model sees the result. This is not translation. Entries are applied in dictionary order, and we found that seven of the 36 keys can never match because an earlier, shorter key is a substring of them and replaces it first. For example, in the Hindi phrase meaning “I am happy”, the short key for “khush” fires first and inserts the English word “happy” but leaves a stray vowel sign behind. The prediction in that example is still sensible, but the defect is real; applying longer keys first would remove it.
C. Wellbeing self-check
The questionnaire has 10 items answered on a four-point frequency scale (0–3) over the last two weeks, in English, Hindi and Marathi. Eight items follow the wording style of PHQ-9 [19] items 1–8 (interest, mood, sleep, energy, appetite, self-view, concentration, restlessness) and two follow GAD-7 [20] items 1–2 (nervousness, uncontrollable worry); the PHQ-9 item on thoughts of self-harm is not included. The total (0–30) is placed in one of five bands (0–4, 5–9, 10–14, 15–19, 20–30) that reuse the PHQ-9 severity cut-points. Because the item set and the 30-point total are new, these bands have no validated clinical meaning, and the page says that it is not a diagnosis.
D. Suggestions
Suggestions are rule-based text in three languages for seven emotions (sadness, anger, fear, nervousness, joy, gratitude, neutral), chosen in a fixed priority order from the predicted labels; other labels, such as grief or disappointment, fall back to generic text unless a covered emotion is also predicted. The wording has not been reviewed by a clinician.
E. Crisis notice
A crisis notice is shown when the raw text contains one of nine English phrases (for example “want to die” or “self-harm”). The classifier has no self-harm label, since GoEmotions has none, and the check does not cover Hindi or Marathi, paraphrases or misspellings. It will also fire on text that merely discusses the topic. It is a minimal safeguard and not a risk assessment.
VIII. SYSTEM DEPLOYMENT
The application is written with Flask [27] and SQLAlchemy over SQLite. The trained bundle is loaded once and reused. Fig. 2 shows the emotion pipeline and Fig. 3 the application components.

Figure 2: Emotion pipeline: normalisation, TF-IDF, 28 one-vs-rest classifiers and a global threshold.

Figure 3: System architecture of MindCare AI.
Accounts and data. Users register with a username (3–40 letters, digits or underscores), an e-mail address and a password of at least eight characters; passwords are stored as Werkzeug hashes. Input is limited to 2,000 characters and 32 KB per request. A saved analysis stores the predicted labels, the scores, the top emotion, the model version and an optional note of up to 300 characters; the text itself is stored only if the user ticks a box. Results are visible only to their owner, and a user can delete one analysis, clear the history or delete the account. State-changing forms use CSRF tokens, and the stateless prediction endpoint is exempt from them. A report page can be printed to PDF from the browser. The self-check result can be e-mailed to the account address when SMTP settings are present in the environment; otherwise nothing is sent.
Interface. Pages cover home, analysis, dashboard, history, detail, report, settings, about and the self-check, with English, Hindi and Marathi interface text selectable per session. The JSON API offers /api/predict, /api/analyses and /api/health. The about page lists the limits of the tool and points to the national Tele-MANAS service [28]; the contact details must be verified before release.
Testing and speed. The project includes 16 automated tests covering the label file and training-set size, model output format and edge inputs (unseen tokens, very long text), the crisis check, language selection, API validation (status codes 422, 400 and 413), authentication, password hashing, per-user isolation, deletion, CSRF enforcement and an end-to-end form flow. In a micro-benchmark of the classifier alone (200 repeated single-comment predictions, one process), a prediction took about 1.8 ms and loading the bundle about 0.45 s; these figures depend on the machine and exclude web-server overhead, so end-to-end latency and memory use have not been benchmarked.
Privacy. All inference runs locally; no text is sent to a third-party AI service. Even so, storing emotional text is sensitive, and any real deployment must follow applicable data-protection law, such as India’s Digital Personal Data Protection Act, 2023 [29].
IX. EXPERIMENTAL RESULTS
A. Official test split
Table 6 lists per-label results on the 5,427 test comments at τ = 0.45. Macro-F1 is 0.445 (macro precision 0.391, macro recall 0.534); micro-F1 is 0.514 with micro precision 0.430 and micro recall 0.638. Subset accuracy is 0.276, Hamming loss 0.0503, and the top-scoring label is a true label for 56.9% of comments. Re-scoring the saved model on the test split reproduced these values exactly.
Table 6: Per-label results on the official test split (sorted by test F1; supp. = test comments with the label; last column = validation F1).
| Label | P | R | F1 | Supp. | Val. F1 |
|---|---|---|---|---|---|
| gratitude | 0.857 | 0.920 | 0.888 | 352 | 0.889 |
| amusement | 0.709 | 0.879 | 0.785 | 264 | 0.746 |
| love | 0.685 | 0.849 | 0.758 | 238 | 0.753 |
| neutral | 0.545 | 0.770 | 0.638 | 1787 | 0.621 |
| remorse | 0.511 | 0.839 | 0.635 | 56 | 0.747 |
| fear | 0.581 | 0.692 | 0.632 | 78 | 0.591 |
| admiration | 0.538 | 0.736 | 0.622 | 504 | 0.661 |
| joy | 0.419 | 0.689 | 0.521 | 161 | 0.533 |
| optimism | 0.437 | 0.629 | 0.515 | 186 | 0.551 |
| sadness | 0.397 | 0.571 | 0.468 | 156 | 0.453 |
| grief | 0.429 | 0.500 | 0.462 | 6 | 0.400 |
| pride | 0.600 | 0.375 | 0.462 | 16 | 0.533 |
| disgust | 0.414 | 0.512 | 0.458 | 123 | 0.372 |
| surprise | 0.361 | 0.518 | 0.426 | 141 | 0.465 |
| desire | 0.379 | 0.470 | 0.419 | 83 | 0.475 |
| anger | 0.343 | 0.520 | 0.414 | 198 | 0.420 |
| curiosity | 0.317 | 0.518 | 0.394 | 284 | 0.393 |
| confusion | 0.262 | 0.536 | 0.352 | 153 | 0.345 |
| disapproval | 0.242 | 0.457 | 0.316 | 267 | 0.345 |
| excitement | 0.236 | 0.456 | 0.311 | 103 | 0.285 |
| annoyance | 0.236 | 0.409 | 0.300 | 320 | 0.251 |
| embarrassment | 0.289 | 0.297 | 0.293 | 37 | 0.515 |
| realization | 0.232 | 0.372 | 0.286 | 145 | 0.213 |
| approval | 0.211 | 0.390 | 0.274 | 351 | 0.314 |
| caring | 0.203 | 0.363 | 0.261 | 135 | 0.355 |
| disappointment | 0.189 | 0.272 | 0.223 | 151 | 0.241 |
| nervousness | 0.227 | 0.217 | 0.222 | 23 | 0.316 |
| relief | 0.100 | 0.182 | 0.129 | 11 | 0.176 |
| Macro (28 labels) | 0.391 | 0.534 | 0.445 | – | 0.463 |
Labels with explicit cue words are recognised well: gratitude (F1 0.888), amusement (0.785) and love (0.758), followed by neutral (0.638), remorse (0.635) and fear (0.632). The weakest labels are relief (0.129, 11 test comments), nervousness (0.222), disappointment (0.223), caring (0.261), approval (0.274), realization (0.286) and annoyance (0.300). Labels that depend on context or on tone, such as approval and realization, are hard for a bag-of-words model. Rare labels have very few test comments (grief 6, relief 11, pride 16), so their F1 values carry large uncertainty. Fig. 4 compares validation and test F1 for every label.

Figure 4: Per-label F1 on the validation and test splits at threshold 0.45.
B. Error structure
The model recalls many true labels (0.638) but at low precision (0.430): it predicts 1.73 labels per comment on average against 1.17 annotated, and macro recall exceeds macro precision (0.534 against 0.391). Neutral is predicted for 46.6% of comments although it is annotated for 32.9%, and 31.6% of comments whose annotation is not neutral receive a neutral prediction, which is the main way in which real emotion can be missed. The balanced class weights push scores of rare labels up, which lifts recall and costs precision. Fig. 5 shows only a weak, non-significant relation between a label’s test support and its F1 (Spearman ρ = 0.32, p = 0.10), which suggests that how distinctive a label’s wording is matters more than its frequency; this is a hypothesis, not a result.

Figure 5: Test F1 against test support for the 28 labels.
C. Validation versus test, and threshold sensitivity
Table 7: Summary on the validation and test splits (τ = 0.45).
| Metric | Validation | Test |
|---|---|---|
| Macro-F1 | 0.4628 | 0.4451 |
| Macro precision / recall | 0.409 / 0.545 | 0.391 / 0.534 |
| Micro-F1 | 0.5189 | 0.5140 |
| Micro precision / recall | 0.441 / 0.629 | 0.430 / 0.638 |
| Subset accuracy | 0.2879 | 0.2762 |
| Hamming loss | 0.0490 | 0.0503 |
Validation and test scores are close (macro-F1 0.463 against 0.445), as expected when both splits are random samples of the same Reddit data and τ was chosen on validation data. This agreement says nothing about other kinds of text. Fig. 6 shows that macro-F1 changes little for τ between 0.40 and 0.65 (validation 0.455–0.463, test 0.440–0.445), whereas micro-F1 rises from 0.502 at τ = 0.40 to 0.526 at 0.55 as precision improves; the choice of τ therefore trades emotion coverage against false alarms, and a single global τ is a simplification.

Figure 6: Effect of the global threshold on validation and test F1.
D. Qualitative probes
Table 8 shows hand-written probes passed through the deployed pipeline. They are illustrations and not an evaluation; the Hindi and Marathi inputs were written in Devanagari and are given here in transliteration.
Table 8: Qualitative probes (scores rounded to two decimals).
| Input | Top scores (model) | Predicted |
|---|---|---|
| Thank you so much, this really helped me! | gratitude 1.00, admiration 0.25, approval 0.12 | gratitude |
| I am so nervous about my exam tomorrow | nervousness 0.99, confusion 0.25, excitement 0.23 | nervousness |
| My grandmother passed away last week and I miss her | sadness 0.97, grief 0.68, neutral 0.50 | sadness, grief, neutral |
| I can’t believe he lied to me again, I am furious | surprise 0.91, anger 0.31, confusion 0.21 | surprise (anger missed) |
| Not bad at all, honestly | disapproval 0.97, disgust 0.86, annoyance 0.71 | three negative labels (negation missed) |
| Hindi: “mujhe khushi hai” (I am happy) | without lexicon: neutral 0.74; with lexicon: joy 1.00, excitement 0.99, gratitude 0.54 | joy, excitement, gratitude |
| Marathi: “mala vait vatatay” (I feel bad) | with lexicon: disappointment 1.00, disgust 0.97, sadness 0.96 | six labels, incl. disgust and annoyance |
| Crying-face emoji alone | without lexicon: neutral 0.74; with lexicon: sadness 1.00, disappointment 0.46 | sadness, disappointment |
| I want to die | desire 0.97, sadness 0.91, anger 0.85 | desire, sadness, anger, disappointment; crisis notice shown |
The probes confirm the pattern seen in the metrics: explicit emotion words work, whereas negation (“not bad”), ambiguous wording and mixed feelings do not. Without the lexicon, every unmatched Hindi, Marathi or emoji input yields the same output (neutral 0.74), so the model has no knowledge of those inputs at all. The last row shows why a separate crisis check is needed: the classifier reads the sentence as desire, sadness and anger.
DISCUSSION
The classifier recognises emotions that have explicit cue words and is weak on labels that need context. It over-predicts, and its largest practical error is calling emotional text neutral. The macro-F1 is not far below the roughly 0.46 reported for BERT on this dataset, but this closeness should not be read as equivalence, since we did not re-run BERT and our figures come from a single run. Plausible contributors to the weak labels are label noise and annotator disagreement, the lack of word order beyond bigrams, and a single global threshold; we have not isolated any of them.
The larger concern is transfer. The model was trained on public Reddit comments, whereas MindCare AI is meant for personal reflection, possibly in Hindi or Marathi, possibly mixed with English. Predicting a person’s emotion from a sentence is also not the same as assessing their mental health [21]. For these reasons the interface frames every output as an awareness signal, and the self-check, suggestions and crisis notice are described as unvalidated aids. Measuring how the system behaves on its intended text and languages is the main open question.
XI. LIMITATIONS
XII. FUTURE WORK
The most important next step is a stronger evaluation: several seeds or bootstrap intervals, the baselines and ablations in Table 5, and training on de-duplicated data.
CONCLUSION
We presented MindCare AI, a documented pipeline and web application for multi-label emotion detection from short text, and evaluated its classifier on the official GoEmotions test split. The model reaches a macro-F1 of 0.445 and a micro-F1 of 0.514, recognises explicit emotions such as gratitude, amusement and love well, over-predicts, and fails on relief, nervousness, disappointment and context-dependent labels. The multilingual layer, self-check, suggestions and crisis notice are working design components that have not been validated. The results are a starting point and not evidence of generalisation: they come from one run on one dataset without baselines. Until stronger models, multilingual evaluation, security hardening and expert review are in place, the system should be used only as a research prototype and an awareness aid.
DECLARATIONS
Data availability. GoEmotions is publicly available from its authors [5]. The evaluation uses its official train, dev and test splits; the project code, trained bundle and the per-label results file (reports/evaluation_v2.json) are in the project archive.
Use of AI tools. The application does not call any external AI service at run time; all inference runs locally with the bundled scikit-learn model.
REFERENCES
Pratiksha Mogal, Vishakha Lonkar, Disha Nimbalkar, Vedansh Soni, Dr. Rama Bansode, MindCare AI: Multi-Label Emotion Detection from Text for Mental-Wellbeing Awareness with English, Hindi and Marathi Support, Int. J. Sci. R. Tech., 2026, 3 (11), 196-208. https://doi.org/10.5281/zenodo.23942283
10.5281/zenodo.23942283