Skip to content

CSV Data Validator is a tool to validate csv file. It parse csv and validate the data with .hdr(csv meta data) before ingestion to Data Lake. It checks data file availability for every day load and validate data with respective meta data like File Size, Checksum, Delimiter, Record count etc. It ensure landed data conformity before give go ahead …

Notifications You must be signed in to change notification settings

Ashishkr007/CSVDataValidator

Repository files navigation

CSVDataValidator

CSV Data Validator is a tool to validate csv file. It parse csv and validate the data with .hdr(csv meta data) before ingestion to Data Lake. It checks data file availability for every day load and validate data with respective meta data like File Size, Checksum, Delimiter, Record count etc. It ensure landed data conformity before give go ahead for data ingestion to Data Lake. It generate complete stats or error log.

Property File

CSVDataValidator/src/main/resources/dv_config.properties

DelimiterCount=36 Delimiter=, Source=Source001 HeaderFileFormat=.hdr CheckSum=MD5 Headerlines=0 Date=20180202 Auditpath=E:\StudyDoc\ProjectsWS\JavaWS\CSVDataValidator\testDir\Logging DataFeed=Feed001 HeaderDelimiter=, SourcePath=E:\StudyDoc\ProjectsWS\JavaWS\CSVDataValidator\testDir\SourceData\Source001 DataFileFormat=.csv DateFormat=yyyyMMdd

Test Data and HDR file

CSVDataValidator/testDir/SourceData/Source001/Feed001/Feed001TestData_20180202.HDR CSVDataValidator/testDir/SourceData/Source001/Feed001/Feed001TestData_20180202.csv

About

CSV Data Validator is a tool to validate csv file. It parse csv and validate the data with .hdr(csv meta data) before ingestion to Data Lake. It checks data file availability for every day load and validate data with respective meta data like File Size, Checksum, Delimiter, Record count etc. It ensure landed data conformity before give go ahead …

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages