im having trouble displaying alot of Data. Its 4 CSV Files with 40.000 Rows and 6 Colums.
Date | Time | Value 1 | Value2 etc
Currently im spliting the Data with Occurance by Date and allow the user to select a Day of the data.
It works but this is ofcourse not working with exeptable performance.
i need the Data from the Huge Files to be grouped by Date.
Currently i have 2 Possible solutions.
Render out All Graphs per Day as dds files ( not best solution no Data Mapping possible / no animations )
Split the Huge CSV into many CSV Files, One for each Day in the Data.
This would result in a Folder per File with ~400 small CSV Files with about 100-600 rows.
With these i would just select the File string and load only one of them for the Day selected.
So my questions are: is the a better solution im not seing? All Database / Format options ive found would also result in Heavy String Operations.
Does anyone know a tool or Script to separate the CSV File into many files grouped by Date?
I Could do this in VVVV…
Would the Solution spliting it into many small CSV Files make sense or would reading them result in bad Performance?
Just asking to make sure im not wasting time to find out my solutions make no sense…
Screenshot to explain my filtering