• 开发板介绍

MAX78000FTHR为快速开发平台，帮助工程师利用MAX78000 Arm^® Cortex^® M4F处理器快速实施超低功耗、人工智能(AI)方案，器件集成卷积神经网络加速器。

高集成度的Maxim Integrated MAX78000FTHR 开发平台是基于 MAX78000 Arm Cortex M4F 处理器，并且集成卷积神经网络加速器，面向边缘运行的 AI 应用（例如家庭、工业环境或 车辆）。 正如 Maxim Integrated 所解释的，“MAX78000FTHR 提供了一个功耗优化的灵活平台，用于快速概念验证和早期软件开发，从而加快产品上市时间。”

• 开发板上手

下载开发IDE Eclipse 地址：

https://www.analog.com/en/design-center/evaluation-hardware-and-software/software/software-download?swpart=SFW0010820A

官方资料

https://github.com/MaximIntegratedAI/MaximAI_Documentation

https://github.com/MaximIntegratedAI/MaximAI_Documentation/blob/master/MAX78000_Feather/README.md

- Hello world示例

按照如下路径 "New"->"Maxim Microcontrollers"选择

输入项目名称，然后单击“下一步”。

“选择项目配置”对话框可以配置如下：

注意，默认情况下，所有示例都以EVKIT为目标。要将目标更改为FTHR，必须通过BOARD=FTHR_RevA进行设置。

这可以在Eclipse中按项目完成，如下所示：

编译 与下载后  可以看到 灯效闪烁的demo

这说明，开发板正常 可以继续进行后续的开发。

• 项目介绍

我们要完成的项目是语音照相机。用语音的形式，控制照相机的开启，并保存照片到SD卡。

项目可以分为三大模块。语音模块，照相模块，保存模块。

语音模块：语音模块，需要采集语音数据，训练模型，导入模型 是整个项目里面最为复杂的部分。

拍照模块：调用拍照接口，底层调用摄像头外设，进行拍照，返回图像buffer数据

保存模块：接受来自拍照模块的buffer数据，保存为文件形式。保存模块需要构建文件系统FATFS，用来文件管理；

• 项目框图

 - 模块思路

   语音模块：

采集语音数据

训练语音数据

生成模型 导入识别系统

拍照模块：

驱动摄像头外设，对外封装接口，提供拍照与最终拍照的buffer

保存模块：

构建FATFS文件系统，提供文件管理，并对外提供文件保存的接口。

 - 关键代码

拍照接口

img_data_t capture_img(uint32_t w, uint32_t h, pixformat_t pixel_format, dmamode_t dma_mode,
                       int dma_channel)
{
    img_data_t img_data;

    // 1. Configure the camera with the 'camera_setup' function.
    // Image dimensions should fall within the limitations
    // of the camera hardware and MCU SRAM limits.  In this simple capture mode the
    // camera.h drivers will allocate an SRAM buffer whose size is equal to
    // width * height * bytes_per_pixel.  See camera.c for implementation details.
    printf("Configuring camera\n");
    int ret = camera_setup(w, // width
                           h, // height
                           pixel_format, // pixel format
                           FIFO_FOUR_BYTE, // FIFO mode (four bytes is suitable for most cases)
                           dma_mode, // DMA (enabling DMA will drastically decrease capture time)
                           dma_channel); // Allocate the DMA channel retrieved in initialization

    // Error check the setup function.
    if (ret != STATUS_OK) {
        printf("Failed to configure camera!  Error %i\n", ret);
        img_data.raw = NULL;
        return img_data;
    }

    // 2. Start the camera capture with the 'camera_start_capture_image()' function.
    // This will 'launch' the image capture, but is non-blocking.  The CameraIF peripheral's
    // hardware handles the capture in the background.
    printf("Capturing image\n");
    MXC_TMR_SW_Start(MXC_TMR0);
    camera_start_capture_image();

    // 3. Wait until the image is fully received.
    while (!camera_is_image_rcv()) {}
    int elapsed_us = MXC_TMR_SW_Stop(MXC_TMR0);
    printf("Done! (Took %i us)\n", elapsed_us);

    // 4. Retrieve details of the captured image from the camera driver with the
    // 'camera_get_image' function.  We don't need to copy any image data here, we'll
    // just retrieve a pointer to the camera driver's internal SRAM buffer.
    img_data.pixel_format = camera_get_pixel_format(); // Retrieve the pixel format of the image
    camera_get_image(&img_data.raw, &img_data.imglen, &img_data.w,
                     &img_data.h); // Retrieve info using driver function.
    printf("Captured %ux%u %s image to buffer location 0x%x (%i bytes)\n", img_data.w, img_data.h,
           img_data.pixel_format, img_data.raw, img_data.imglen);

    // 5. At this point, "img_data.raw" is pointing to the fully captured
    // image data, and all the info needed to decode it has been collected.
    return img_data;
}

拍照保存接口

void save_stream_sd(cnn_img_data_t img_data, char *file)
{
    if (img_data.raw != NULL) { // Image data will be NULL if something went wrong during streaming
        // If file is NULL, find the next available file to save to.
        if (file == NULL) {
            int i = 0;

            for (;;) {
                // We'll use the global sd_filename buffer for this and
                // try to find /raw/imgN while incrementing N.
                memset(sd_filename, '\0', sizeof(sd_filename));
                snprintf(sd_filename, sizeof(sd_filename), "/raw/img%u", i++);
                sd_err = f_stat(sd_filename, &sd_fno);
                if (sd_err == FR_NO_FILE) {
                    file = sd_filename; // Point 'file' to the available path string
                    break;
                } else if (sd_err != FR_OK) {
                    printf("Error while searching for next available file: %s\n",
                           FR_ERRORS[sd_err]);
                    break;
                }
            }
        }

        sd_err = f_open(&sd_file, (const TCHAR *)file, FA_WRITE | FA_CREATE_NEW);

        if (sd_err != FR_OK) {
            printf("Error opening file: %s\n", FR_ERRORS[sd_err]);
        } else {
            printf("Saving image to %s\n", file);

            MXC_TMR_SW_Start(MXC_TMR0);

            // Write image info as the first line of the file.
            clear_serial_buffer();
            snprintf(g_serial_buffer, SERIAL_BUFFER_SIZE,
                     "*IMG* %s %i %i %i\n", // Format img info into a new-line terminated string
                     img_data.pixel_format, img_data.imglen, img_data.w, img_data.h);

            unsigned int wrote = 0;
            sd_err = f_write(&sd_file, g_serial_buffer, strlen(g_serial_buffer), &wrote);
            if (sd_err != FR_OK || wrote != strlen(g_serial_buffer)) {
                printf("Failed to write header to file: %s\n", FR_ERRORS[sd_err]);
            }
            clear_serial_buffer();

            // Similar to streaming over UART, a secondary buffer is needed to
            // save the raw data to the SD card since the CNN data SRAM is non-contiguous.
            // Raw image data is written row by row.
            uint32_t *cnn_addr = img_data.raw;
            uint8_t *buffer = (uint8_t *)malloc(img_data.w);
            for (int i = 0; i < img_data.imglen; i += img_data.w) {
                cnn_addr = read_bytes_from_cnn_sram(buffer, img_data.w, cnn_addr);
                sd_err = f_write(&sd_file, buffer, img_data.w, &wrote);

                if (sd_err != FR_OK || wrote != img_data.w) {
                    printf("Failed to image data to file: %s\n", FR_ERRORS[sd_err]);
                }

                // Print progress %
                if (i % (img_data.w * 32) == 0) {
                    printf("%.1f%%\n", ((float)i / img_data.imglen) * 100.0f);
                }
            }
            free(buffer);
            int elapsed = MXC_TMR_SW_Stop(MXC_TMR0);
            printf("Finished (took %ius)\n", elapsed);
        }
        f_close(&sd_file);
    }
}

命令行调试接口

void service_console()
{
    // Check for any incoming serial commands
    cmd_t cmd = CMD_UNKNOWN;
    if (recv_cmd(&cmd)) {
        // Process the received command...

        if (cmd == CMD_UNKNOWN) {
            printf("Uknown command '%s'\n", g_serial_buffer);
        } else if (cmd == CMD_HELP) {
            print_help();
        } else if (cmd == CMD_RESET) {
            // Issue a soft reset
            MXC_GCR->rst0 |= MXC_F_GCR_RST0_SYS;
        } else if (cmd == CMD_IMGRES) {
            sscanf(g_serial_buffer, "imgres %u %u", &g_app_settings.imgres_w,
                   &g_app_settings.imgres_h);
            printf("Set image resolution to width %u, height %u\n", g_app_settings.imgres_w,
                   g_app_settings.imgres_h);
        } else if (cmd == CMD_CAPTURE) {
            // Perform a blocking image capture with the current camera settings.
            img_data_t img_data = capture_img(g_app_settings.imgres_w, g_app_settings.imgres_h,
                                              g_app_settings.pixel_format, g_app_settings.dma_mode,
                                              g_app_settings.dma_channel);

#ifdef CAMERA_BAYER
            uint8_t *bayer_data = (uint8_t *)malloc(img_data.w * img_data.h * 2);
            if (bayer_data != NULL) {
                MXC_TMR_SW_Start(MXC_TMR0);
                if (g_app_settings.bayer_function == BAYER_FUNCTION_PASSTHROUGH) {
                    bayer_passthrough(img_data.raw, img_data.w, img_data.h, (uint16_t *)bayer_data);
                } else if (g_app_settings.bayer_function == BAYER_FUNCTION_BILINEAR) {
                    bayer_bilinear_demosaicing(img_data.raw, img_data.w, img_data.h,
                                               (uint16_t *)bayer_data);
                }

                img_data.raw = bayer_data;
                img_data.imglen *= 2;
                img_data.pixel_format = (uint8_t *)"RGB565";
                unsigned int elapsed_us = MXC_TMR_SW_Stop(MXC_TMR0);
                printf("Debayering complete. (Took %u us)\n", elapsed_us);
            } else {
                printf("Failed to allocate memory for debayering!\n");
                return;
            }
#endif

            transmit_capture_uart(img_data);

#ifdef CAMERA_BAYER
            free(bayer_data);
#endif
        } else if (cmd == CMD_STREAM) {
            // Perform a streaming image capture with the current camera settings.
            cnn_img_data_t img_data = stream_img(g_app_settings.imgres_w, g_app_settings.imgres_h,
                                                 g_app_settings.pixel_format,
                                                 g_app_settings.dma_channel);

            transmit_stream_uart(img_data);

            // Disable the CNN when unused to preserve power.
            cnn_disable();
        } else if (cmd == CMD_SETREG) {
            // Set a camera register
            unsigned int reg;
            unsigned int val;
            // ^ Declaring these as unsigned ints instead of uint8_t
            // avoids some issues caused by type-casting inside of sscanf.

            sscanf(g_serial_buffer, "%s %u %u", cmd_table[cmd], &reg, &val);
            printf("Writing 0x%x to camera reg 0x%x\n", val, reg);
            camera_write_reg(reg, val);
        } else if (cmd == CMD_GETREG) {
            // Read a camera register
            unsigned int reg;
            uint8_t val;
            sscanf(g_serial_buffer, "%s %u", cmd_table[cmd], &reg);
            camera_read_reg(reg, &val);
            snprintf(g_serial_buffer, SERIAL_BUFFER_SIZE, "Camera reg 0x%x=0x%x", reg, val);
            send_msg(g_serial_buffer);
#ifdef CAMERA_BAYER
        } else if (cmd == CMD_SETDEBAYER) {
            char buffer[20] = "\0";
            sscanf(g_serial_buffer, "%s %s", cmd_table[cmd], buffer);
            if (!strcmp("passthrough", buffer)) {
                g_app_settings.bayer_function = BAYER_FUNCTION_PASSTHROUGH;
                printf("Set %s\n", buffer);
            } else if (!strcmp("bilinear", buffer)) {
                g_app_settings.bayer_function = BAYER_FUNCTION_BILINEAR;
                printf("Set %s\n", buffer);
            } else {
                printf("Unknown debayering function '%s'\n", buffer);
            }
#endif
#ifdef SD
        } else if (cmd == CMD_SD_MOUNT) {
            // Mount the SD card
            sd_err = sd_mount();
            if (sd_err == FR_OK) {
                sd_get_size();

                printf("Disk Size: %u bytes\n", sd_sectors_total / 2);
                printf("Available: %u bytes\n", sd_sectors_free / 2);
            }
        } else if (cmd == CMD_SD_UNMOUNT) {
            sd_unmount();
        } else if (cmd == CMD_SD_CWD) {
            // Get the current working directory
            sd_err = sd_get_cwd();
            if (sd_err == FR_OK) {
                printf("%s\n", sd_cwd);
            }
        } else if (cmd == CMD_SD_CD) {
            // Change the current working directory
            char *b = (char *)malloc(SERIAL_BUFFER_SIZE);
            sscanf(g_serial_buffer, "cd %s", b); // Parse the target directory
            sd_cd(b);
            free(b);
        } else if (cmd == CMD_SD_LS) {
            sd_ls();
        } else if (cmd == CMD_SD_MKDIR) {
            // Make a directory
            char *b = (char *)malloc(SERIAL_BUFFER_SIZE);
            sscanf(g_serial_buffer, "mkdir %s", b); // Parse the directory name
            sd_mkdir(b);
            free(b);
        } else if (cmd == CMD_SD_RM) {
            // Remove an item
            char *b = (char *)malloc(SERIAL_BUFFER_SIZE);
            sscanf(g_serial_buffer, "rm %s", b); // Parse the item name
            sd_rm(b);
            free(b);
        } else if (cmd == CMD_SD_TOUCH) {
            // Create a new empty file
            char *b = (char *)malloc(SERIAL_BUFFER_SIZE);
            sscanf(g_serial_buffer, "touch %s", b); // Parse the filepath
            sd_touch(b);
            free(b);
        } else if (cmd == CMD_SD_WRITE) {
            // Write a string to a file
            char *b = (char *)malloc(SERIAL_BUFFER_SIZE);
            sscanf(g_serial_buffer, "write %s \"%[^\"]", sd_filename,
                   b); // \"$[^\"] captures everything between two quotes ""
            sd_write_string(sd_filename, b);
            free(b);
        } else if (cmd == CMD_SD_CAT) {
            // Print the contents of a file
            sscanf(g_serial_buffer, "cat %s", (char *)sd_filename); // Parse the target file
            sd_cat(sd_filename);
        } else if (cmd == CMD_SD_SNAP) {
            // Stream and save an image to the SD card
            int args = sscanf(g_serial_buffer, "snap %s", (char *)sd_filename);

            cnn_img_data_t img_data = stream_img(g_app_settings.imgres_w, g_app_settings.imgres_h,
                                                 g_app_settings.pixel_format,
                                                 g_app_settings.dma_channel);

            if (args == 1) {
                save_stream_sd(img_data, sd_filename);
            } else {
                save_stream_sd(img_data, NULL);
            }

            cnn_disable();
#endif // #ifdef SD
        }

        // Clear the serial buffer for the next command
        clear_serial_buffer();
    }
}
#endif // #ifdef CONSOLE

实现结果

最终实现了 指令版本的拍照与保存 可以理解是实现了拍照与保存

语音数据采集部分，时间关系，采集部分，未能达到预期（会继续完善到完成）

遇到的主要问题

文件系统的构建

文件系统有很多，但是适合的不太多，找了市面上好几个文件系统，最终确认FATFS

拍照接口与保存接口的适配

可以通过参数设置

心得与感想

很不错的项目，可实现的功能很多 扩展性很强。