【仿真建模-anylogic】数据源组件

07-19 1368阅读

Author：赵志乾
Date：2024-07-16
Declaration：All Right Reserved！！！

1. 简介

仿真模型依赖的数据源通常有Excel文件、MySQL数据库两种；针对小数量、大数据量以及是否允许外部依赖等场景设计了一套通用数据源组件；该套数据源组件支持3种数据源：

（图片来源网络，侵删）

小数据量且无外部依赖：ExcelDataSource
大数据量且无外部依赖：MultiExcelDataSource

允许外部依赖：MySqlDataSource

数据存取操作均通过接口IDataSource进行，依据实际场景不同，切换不同数据源实现即可；

2. 抽象数据源接口

public interface IDataSource {
    // taskId为一次仿真的唯一标识
    // containerName为数据库表名或者Sheet页名称
    // items为要存储的数据
    // clazz为数据类信息
     void store(String taskId, String containerName, List items, Class clazz);
     List query(String taskId, String containerName, Class clazz);
}

3. ExcelDataSource

ExcelDataSource针对于小数据量场景，单个Excel即可存储所有数据；

public class ExcelDataSource implements IDataSource {
    // excel文件路径
    private final String path;
    public ExcelDataSource(String path) {
        this.path = path;
    }
    public ExcelDataSource() {
        this("");
    }
    @Override
    public  void store(String taskId, String containerName, List items, Class clazz) {
        EasyExcelUtil.write(path + taskId, "data.xlsx", containerName, items, clazz);
    }
    @Override
    public  List query(String taskId, String containerName, Class clazz) {
        List result = new ArrayList();
        Path directoryPath = Paths.get(path + taskId);
        try (Stream paths = Files.list(directoryPath)) {
            paths.forEach(file -> {
                String fileName = file.getFileName().getFileName().toString();
                if(fileName.endsWith("xlsx")){
                    result.addAll(EasyExcelUtil.read(path + taskId, fileName, containerName, clazz));
                }
            });
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
        return result;
    }
}

4. MultiExcelDataSource

MultiExcelDataSource针对大数据量而又不希望引入外部依赖的场景，其输入输出支持多Excel文件，以文件名数字后缀进行数据的切分；

public class MultiExcelDataSource implements IDataSource {
    private final String path;
    private final IDataSource excelDataSource;
    public MultiExcelDataSource(String path) {
        this.path = path;
        excelDataSource = new ExcelDataSource(path);
    }
    public MultiExcelDataSource() {
        this("");
    }
    @Override
    public synchronized  void store(String taskId, String containerName, List items, Class clazz) {
        int batchSize = 1000;
        int fileNum = (items.size() + batchSize - 1) / batchSize;
        for (int index = 0; index  
5. MySqlDataSource  
        MySqlDataSource适用于大数量场景； 
@Data
@AllArgsConstructor
public class MySqlDataSource implements IDataSource {
    private final String url;
    private final String userName;
    private final String password;
    private final static int batchSize = 500;
    @Override
    public synchronized  void store(String taskId, String containerName, List items, Class clazz) {
        Field[] fields = clazz.getDeclaredFields();
        Map columnToFieldMap = getColumToFieldMap(fields, clazz);
        Connection connection = null;
        PreparedStatement preparedStatement = null;
        try {
            connection = DriverManager.getConnection(url, userName, password);
            connection.setAutoCommit(false);
            StringBuilder sql = new StringBuilder("INSERT INTO ");
            sql.append(containerName).append("(task_id,");
            List columns = new ArrayList(columnToFieldMap.keySet());
            for (int index = 0; index ").append(" ? ")
                    .append(" order by id asc")
                    .append(" limit ").append(batchSize);
            System.out.println(sql.toString());
            preparedStatement = connection.prepareStatement(sql.toString());
            for (int index = 0; index  
6. 使用说明