MMLab-dataset

数据分析工具

这里写目录标题

• 数据分析工具
• • dataset_analysis.py
  • • 数据可视化分析
    • benchmark.py
    • browse_coco_json.py
    • browse_dataset.py
    • Optimize_anchors

      mmyolo、mmsegmentation等提供了数据集分析工具

      

      dataset_analysis.py

      数据采用coco格式数据

      根据配置文件分析全部数据类型或指定类型的Bbox_num、bbox_wh\bbox_wh_ratio、bbox_area

      

      

      示例数据采用的是讯飞X光安检物品监测数据集，通过结果可以看出Knife、wrench、powerbank等小物品的数据相对较少，Knife类别最少，存在显著的类别不平衡问题。

      数据可视化分析

      • bbox_area

        

      • bbox_ratio

        

      • bbox_wh

        

        benchmark.py

        测试模型性能：推理速度

        !python /root/mmyolo/tools/analysis_tools/browse_coco_json.py --data-root /root/autodl-tmp/train --img-dir /root/autodl-tmp/train/images/ --ann-file /root/autodl-tmp/train/annotations/instances_train2014.json

        browse_coco_json.py

        将数据集与标签进行可视化

        browse_dataset.py

        -将数据可视化保存输出到文件夹下,包含两种模式

        -m：‘original’, ‘transformed’, ‘pipeline’

        ‘original’：金输出原始图像

        ‘transformed’：输出变换后的图像

        ‘pipeline’：输出数据增流各个阶段的图像

        Optimize_anchors

        通过分析数据，优化先验anchor的设置，仅支持YOLOAnchorGenerator

        “”"Optimize anchor settings on a specific dataset.

        This script provides three methods to optimize YOLO anchors including k-means

        anchor cluster, differential evolution and v5-k-means. You can use

        --algorithm k-means, --algorithm differential_evolution and

        --algorithm v5-k-means to switch those methods.

        Example:

        Use k-means anchor cluster::
            python tools/analysis_tools/optimize_anchors.py ${CONFIG} \
            --algorithm k-means --input-shape ${INPUT_SHAPE [WIDTH HEIGHT]} \
            --out-dir ${OUT_DIR}
        Use differential evolution to optimize anchors::
            python tools/analysis_tools/optimize_anchors.py ${CONFIG} \
            --algorithm differential_evolution \
            --input-shape ${INPUT_SHAPE [WIDTH HEIGHT]} \
            --out-dir ${OUT_DIR}
        Use v5-k-means to optimize anchors::
            python tools/analysis_tools/optimize_anchors.py ${CONFIG} \
            --algorithm v5-k-means \
            --input-shape ${INPUT_SHAPE [WIDTH HEIGHT]} \
            --prior_match_thr ${PRIOR_MATCH_THR} \
            --out-dir ${OUT_DIR}
        

        该工具默认调用gpu进行数据计算,算法名称还有个小bug，需要注意一下

         if args.algorithm == 'k-means':
              optimizer = YOLOKMeansAnchorOptimizer(
                  dataset=dataset,
                  input_shape=input_shape,
                  device=args.device,
                  num_anchor_per_level=num_anchor_per_level,
                  iters=args.iters,
                  logger=logger,
                  out_dir=args.out_dir)
          elif args.algorithm == 'DE':
              optimizer = YOLODEAnchorOptimizer(
                  dataset=dataset,
                  input_shape=input_shape,
                  device=args.device,
                  num_anchor_per_level=num_anchor_per_level,
                  iters=args.iters,
                  logger=logger,
                  out_dir=args.out_dir)
          elif args.algorithm == 'v5-k-means':
              optimizer = YOLOV5KMeansAnchorOptimizer(
                  dataset=dataset,
                  input_shape=input_shape,
                  device=args.device,
                  num_anchor_per_level=num_anchor_per_level,
                  iters=args.iters,
                  prior_match_thr=args.prior_match_thr,
                  mutation_args=args.mutation_args,
                  augment_args=args.augment_args,
                  logger=logger,
                  out_dir=args.out_dir)
          else:
              raise NotImplementedError(
                  f'Only support k-means and differential_evolution, '
                  f'but get {args.algorithm}')