Software Engineering Stack Exchange
Software Engineering Stack Exchange

Questions tagged [csv]

28 questions
46
votes
2 answers

Should UTF-8 CSV files contain a BOM (byte order mark)?

Our line-of-business software allows the user to save certain data as CSV. Since there are a lot of different formats (all called "CSV") in use in the wild, we are tying to decide what the "default format" should look like. Regarding line/field…
Heinzi
  • 9,646
  • 3
  • 46
  • 59
27
votes
5 answers

Can the csv format be defined by a regex?

A colleague and I have recently argued over whether a pure regex is capable of fully encapsulating the csv format, such that it is capable of parsing all files with any given escape char, quote char, and separator char. The regex need not be…
Spencer Rathbun
  • 3,576
  • 1
  • 21
  • 28
22
votes
6 answers

Is CSV a good alternative to XML and JSON?

Is CSV considered a good option against XML and JSON for programming languages? I generally use XML and JSON (or sometimes a plain text file) as flat file storage. However, recently I came across an CSV implementation in PHP. I generally have seen…
Vishwas
  • 1,871
  • 4
  • 15
  • 15
8
votes
4 answers

CSV file generation using Java

We have a requirement in our project to generate a big CSV file every 2 hours using a Java program. This file will have around 60,000 lines (around 120 characters per line). I am not sure about the size yet. I would like to know if I would run into…
java_mouse
  • 2,627
  • 15
  • 23
5
votes
2 answers

Multi-level validation in C#

I have a Console project reads inputs from CSV file and tries to save them to database. For that, I created a class Person that maps a CSV row. The CSV file has two columns Name and Age. Person class is like. class Person { public string Name; …
Mhd
  • 195
  • 1
  • 7
5
votes
4 answers

Best way of validating Class properties

Background: I have a CSV file, which I need to ready and validate each element in each row and create a collection of a class, having valid data. i.e CSV File looks like: EmpID,FirstName,LastName,Salary 1,James,Help,100000 …
user2697452
  • 153
  • 1
  • 1
  • 3
4
votes
5 answers

Import large csv files

I've been tasked to query 2 large csv files which are roughly 1 GB in size each. The files contain related data, so file one may contain a list of order numbers, Order Dates, etc and the file may contain the order lines, stock codes, quantity,…
Rhodes73
  • 43
  • 1
  • 4
3
votes
1 answer

Meaningful response to the user after his uploaded CSV was processed?

A user uploads a CSV to the server to be inserted into the database and I would like to return a meaningful response to this user. My thoughts : { inserted : x records discarded : y records } I believe also that I should specify what records…
Oleg
  • 181
  • 5
2
votes
3 answers

Back-end solution for pulling from CSV files

I'm building a data visualization that displays COVID information for the United States, at the city, state, and county level. The ultimate source of truth are three CSVs published by the New York Times on Github in this…
InspectorDanno
  • 137
  • 2
2
votes
2 answers

SQL query or C# .net code for csv files import?

My aim here is to find out the best possible and feasible solution for my dilemma. I want to import some csv file (may contain around 50~60K records) into database after some manipulation into the files, manipulation includes comparison such as one…
Shilpa Soni
  • 121
  • 5
2
votes
3 answers

Mapping different XML and CSV feeds

Not sure if this is the right venue to be asking this but here goes. A little background. I'm trying to build an ecommerce app that would allow sellers from other venues--like, amazon and newegg--to import their products using their export/import…
clueless
  • 129
  • 1
1
vote
4 answers

System Design: Efficient way to import hundreds of different CSV formats

OVERVIEW We have multiple data providers who provide us information from a large set of data sources. These end data sources provide data in CSV format. Since, the CSV formats are not consistent across different sources, we currently have a manual…
Rishabh
  • 355
  • 1
  • 2
  • 7
1
vote
1 answer

Storing survey data from csv to xml

I've been tasked with taking a sample of survey results that are stored in CSV and to convert them to XML. I'm still fairly new to XML, but have come up with a couple of layouts I could use and was hoping I could get some advice on what way is best…
screencut
  • 43
  • 5
1
vote
0 answers

how to use Sql query to search text in CSV file and avoid iteration

I have one A File and many input files ,All files have CSV format . I want to match text of one column of all input files with one of the column of A file . I have stored all column value of File A in HashSet for fast search. 1- Now iterate All…
Charlie
  • 119
  • 3
1
vote
2 answers

Why do CSV file formats normally use quoting instead of escaping?

There are a lot of variants of the CSV "standard" (or lack thereof). I've never personally see any that use an escape character (like \) instead of surrounding each field with double quotes. Instead of foo,bar,"foo,bar" it would be foo,bar,foo\,bar.…
poke
  • 570
  • 4
  • 12
1
2