OpenCSV with a manual loop for CSV imports was covered in a previous article, but things change once the row count climbs past a few hundred thousand. Loading everything into memory raises the specter of OutOfMemoryError, and having a single malformed row bring down the entire process is painful operationally.

Spring Batch’s FlatFileItemReader is built specifically for importing large CSV files. Assuming you’re already familiar with the basics of Job/Step/Chunk, this article focuses purely on the CSV-reading implementation.

How it differs from an OpenCSV loop

A manual OpenCSV loop tends to read one line at a time into a list and write everything out at the end, so that list eats up memory as the row count grows. FlatFileItemReader reads and writes in chunks, which keeps memory usage stable, and it lets skip/retry handle malformed rows for you.

Basic FlatFileItemReader configuration

The minimal setup looks like this.

@Bean
public FlatFileItemReader<Product> productItemReader() {
    return new FlatFileItemReaderBuilder<Product>()
        .name("productItemReader")
        .resource(new FileSystemResource("/data/products.csv"))
        .linesToSkip(1)
        .lineMapper(productLineMapper())
        .build();
}

resource() specifies the file, and linesToSkip(1) excludes the header row. Converting each line into an object is handled by lineMapper(). This shows a minimal setup with a hardcoded path — it’s provisional code meant to be replaced with the StepScope version discussed later.

Converting to a POJO with DelimitedLineTokenizer and BeanWrapperFieldSetMapper

Inside lineMapper(), DelimitedLineTokenizer splits each line into a FieldSet, and BeanWrapperFieldSetMapper converts that FieldSet into a POJO.

DelimitedLineTokenizer tokenizer = new DelimitedLineTokenizer();
tokenizer.setNames("code", "name", "price", "stock");
tokenizer.setDelimiter(",");
tokenizer.setQuoteCharacter('"');

BeanWrapperFieldSetMapper<Product> fieldSetMapper = new BeanWrapperFieldSetMapper<>();
fieldSetMapper.setTargetType(Product.class);

DefaultLineMapper<Product> lineMapper = new DefaultLineMapper<>();
lineMapper.setLineTokenizer(tokenizer);
lineMapper.setFieldSetMapper(fieldSetMapper);

The column names passed to setNames() become the FieldSet’s keys, and if they match the property names on the class given to setTargetType(), values are set automatically via the setters. Use setDelimiter() if you want to change the delimiter, and setQuoteCharacter() if you want to quote a column whose value may contain commas. If the number of columns doesn’t match the number of names, you’ll get an IncorrectTokenCountException, so make sure it matches the header’s column count.

Simple type conversions like numbers are handled automatically by Spring’s standard mechanism, but a custom date format like yyyy/MM/dd will trigger a BindingException as-is. In that case, register a PropertyEditor such as CustomDateEditor via setCustomEditors().

Map<Class<?>, PropertyEditor> editors = new HashMap<>();
editors.put(Date.class, new CustomDateEditor(new SimpleDateFormat("yyyy/MM/dd"), true));
fieldSetMapper.setCustomEditors(editors);

Making the chunk-oriented Step resilient with skip/retry

Processing the data you’ve read uses the usual chunk-oriented Step. Since some rows may fail type conversion or the DB connection may drop temporarily, configure skip() and retry() together via faultTolerant().

@Bean
public Step productImportStep(JobRepository jobRepository, PlatformTransactionManager tx,
        FlatFileItemReader<Product> reader, ItemWriter<Product> writer,
        ProductSkipListener skipListener) {
    return new StepBuilder("productImportStep", jobRepository)
        .chunk(500, tx)
        .reader(reader)
        .writer(writer)
        .faultTolerant()
        .skip(FlatFileParseException.class)
        .skipLimit(50)
        .retry(TransientDataAccessException.class)
        .retryLimit(3)
        .listener(skipListener)
        .build();
}

The 500 in chunk(500, tx) is the number of items per transaction. A larger value speeds things up but also widens the rollback scope on failure, so tune it somewhere in the 100–1000 range. Without setting skipLimit(50), you wouldn’t notice even if the entire CSV were corrupted and every row got skipped.

Note that skip() and retry() target different exception classes. CSV parsing failures (FlatFileParseException) are subject to skip, while transient DB connection failures (TransientDataAccessException) are subject to retry, and the two work independently. If TransientDataAccessException still isn’t resolved after retryLimit(3) retries, the row isn’t eligible for skip, so the Step itself fails. Only specify exceptions caused by transient failures in retry() (ItemWriter-side optimizations are outside the scope of this article).

Recording skipped rows with SkipListener

If you leave skipped rows unrecorded, you won’t be able to trace them later, so use a SkipListener to log them and tally the count.

@Component
public class ProductSkipListener {
    private static final Logger log = LoggerFactory.getLogger(ProductSkipListener.class);
    private final AtomicInteger skipCount = new AtomicInteger();

    @OnSkipInRead
    public void onSkipInRead(Throwable t) {
        skipCount.incrementAndGet();
        log.warn("skip: {}", t.getMessage());
    }

    @AfterStep
    public ExitStatus afterStep(StepExecution stepExecution) {
        stepExecution.getExecutionContext().putInt("skipCount", skipCount.get());
        return stepExecution.getExitStatus();
    }
}

@OnSkipInRead increments the counter, and @AfterStep writes it into the ExecutionContext, so you can check the skip count after the Job completes. It’s already registered on the Step above via .listener(skipListener).

Passing the CSV file path via JobParameters and StepScope

In real-world usage, you’ll want to pass a different file on each run, so load JobParameters on the JobLauncher side when launching, and receive it on the Reader side via StepScope.

JobParameters params = new JobParametersBuilder()
    .addString("inputFile", "/data/products_20260930.csv")
    .toJobParameters();
jobLauncher.run(productImportJob, params);

On the receiving side, replace the productItemReader defined earlier with a fixed path with the following StepScope version. Since keeping both in the same Configuration class would cause a duplicate bean name error at application startup, keep only this StepScope version.

@Bean
@StepScope
public FlatFileItemReader<Product> productItemReader(
        @Value("#{jobParameters['inputFile']}") String inputFile) {
    return new FlatFileItemReaderBuilder<Product>()
        .name("productItemReader")
        .resource(new FileSystemResource(inputFile))
        .linesToSkip(1)
        .lineMapper(productLineMapper())
        .build();
}

Without StepScope, this bean would be created just once at application startup, before jobParameters is resolved, causing an error. With StepScope, bean creation is deferred until Step execution, so the passed-in inputFile can be received safely.

Summary

Using FlatFileItemReader and BeanWrapperFieldSetMapper lets you import CSV files with hundreds of thousands of rows more memory-efficiently and safely than a hand-written OpenCSV loop. Sorting malformed rows with skip/retry, and passing the file path at runtime via JobParameters and StepScope, gives you an import Job that can hold up in production.

If you haven’t yet gotten a handle on Job/Step/Chunk, check out the Spring Batch introductory guide as well, and for implementing upload/download APIs with OpenCSV, see the CSV import/export guide.