Plant Recognition

Take photos on ESP-Mosaico to identify flowering and herbaceous plants. The application includes a home page, real-time camera preview, Wi‑Fi network configuration, bilingual Chinese‑English encyclopedia result pages, and a local plant guide, supporting bilingual switching and voice narration.

Feature Highlights

• Photo Identification of Plants: The camera submodule is connected on the left side, providing a 480 × 480 full-screen real-time preview (supports 90° clockwise rotation and remembers orientation). Tap “Photo” to identify flowers, leaves, or fruits.
• Bilingual Chinese‑English Encyclopedia: Identification uses Pl@ntNet to obtain species match score, Chinese name, Latin scientific name, and family classification; in Chinese mode it automatically retrieves a Baidu Baike summary (≈ 300 ms), while in English mode it retrieves an English Wikipedia summary via iNaturalist (both are directly reachable on domestic networks; if not found, it automatically falls back to Pl@ntNet family and synonym classification overview).
• Dual‑Engine Voice Narration: Tap “Narration” on the result page to have the encyclopedia description read aloud (supports a 0–100 volume slider for real‑time adjustment). Online it prefers natural streaming voice (Edge TTS); when the network is limited it automatically and seamlessly falls back to the onboard offline Chinese voice (Xiaoxin).
• On‑board NAND Plant Guide: Photo results do not occupy storage by default; only after tapping “Save” are the high‑resolution snapshot and encyclopedia card stored in the on‑board NAND album. Supports grid browsing, detail replay, voice replay, and single‑image deletion.
• Low‑Power and Thermal Optimization: Leaving the preview page or being idle for 60 seconds automatically shuts down the DVP video stream and puts the camera sensor into hardware sleep (PWDN). The screen dims automatically, Wi‑Fi switches to low‑power mode as needed, and the device remains cool during prolonged idle periods.

Required Hardware

• ESP-Mosaico Main Board (ESP32‑S31, includes a 480 × 480 touch screen, speaker, on‑board NAND)
• Camera Expansion Submodule (inserted into the left expansion slot of the main board)

Usage

1. Network Configuration: Power on to the home page, tap the “leaf” button at the top right to scan and connect to Wi‑Fi (the dot at the bottom right is green for connected, amber for connecting, red for not connected). Tap “EN / 中” to switch between English and Chinese interfaces at any time.
2. Framing and Photo Capture: On the home page, tap “Start” to enter the preview page, align the plant within the frame, then tap “Photo”. If the camera is installed in a different orientation, tap “Rotate” to adjust the image angle; tap “Guide” to view saved plant records (swipe inward from the left or right edge of the screen to return to the previous page).
3. View Encyclopedia and Listen to Narration: After identification, the match score, plant name, family scientific name, and a scrollable encyclopedia text are displayed; tap “Narration” to hear the voice introduction, and tap “Save” to add this identification to the on‑board guide.

Configuration and Source Build

The firmware released on this page already includes a trial Pl@ntNet API Key; after installation, simply connect to Wi‑Fi to start identifying (free quota of 500 requests per day, reset at 08:00 Beijing time, shared among all trial users).

If you need to use your own independent quota, you can apply for a free Key at my.plantnet.org, write it into sdkconfig.secrets.defaults in the project directory, and recompile:

CONFIG_PLANTNET_API_KEY="你的_Pl@ntNet_API_Key"

Build in the ESP-Mosaico workspace and update the device:

python mosaico.py iris system-update --project projects/plant_recognition

License

The application source code is licensed under the Apache-2.0 license. The Noto fonts used in the interface follow the SIL Open Font License 1.1, and the DejaVu/Bitstream Vera fonts follow ui/fonts/LICENSE; Espressif components and offline voice data packages follow their respective licenses.