Can't figure out how to hash check CSV files with SQL DB

  • Zealous
  • Guru
  • Guru
  • User avatar
  • Posts: 1244
  • Loc: Sydney

Post 3+ Months Ago

I have been stuck on this problem for a while now where i have a CSV on a RSS feed and i need to be able to download new ones each day and update the sql database with all the new stuff and remove the old stuff.

I have figured out the downloading of the files but i am very puzzled on how i am going to be able to hash check the content in the csv with whats in the SQL. I have been Googling for a while now but if anyone has any idea's or direction would be much appreciated.
  • Anonymous
  • Bot
  • No Avatar
  • Posts: ?
  • Loc: Ozzuland
  • Status: Online

Post 3+ Months Ago

  • spork
  • Brewmaster
  • Silver Member
  • User avatar
  • Posts: 6252
  • Loc: Seattle, WA

Post 3+ Months Ago

Please explain what you mean by "hash check".
  • Zealous
  • Guru
  • Guru
  • User avatar
  • Posts: 1244
  • Loc: Sydney

Post 3+ Months Ago

Sorry, figured it was a term that could be used in context. I am trying to build up a custom plugin for my shopping site.

First off i have a CSV on RSS which downloads to the local server disk. Then i need to update the database with the new products, keep the existing and delete anything that is not in the list.

I have to run this every 24 hours which the direct connection to the wholesaler CSV file to keep a consistent up to date database of products.

The programmer i hired could not figure it out so i am taking on a mission to figure this puzzle out.
  • ScottG
  • Proficient
  • Proficient
  • ScottG
  • Posts: 477

Post 3+ Months Ago

Isn't hash checking used to check for differences in the files? and not to find where the differences are?

Quick question when you mean keep existing I believe your referring to the database records

As far as deleting db info I tend to stay away from an actual delete and mark a deleted flag. and the csv file matching?

Option 1
Would it be plausible to truncate the table or if you use a flag set all the products flags to deleted and then do an import on the csv file? If the flag method is used set the import to override the product record.

option 2
If this is an issue you could Do the import first and then after that is completed use PHP to turn the csv into and array and check the database records against the csv file to find files that exists in the database but not in the csv file and perform the delete on them.

option 3
This is like option 2 but without the import. Turn the csv into an array to check to see in the product exists in the database but not in the csv do a delete. Check to see in the product exists in the csv but not in the database then preform and add.

I just noticed that you didn't mention what server side type script your are running my thoughts are using php but I sure there are options if you are running ASP or something else. but if you are using php this will help you get the csv into an array
http://php.net/manual/en/function.str-getcsv.php

Post Information

  • Total Posts in this topic: 4 posts
  • Users browsing this forum: No registered users and 99 guests
  • You cannot post new topics in this forum
  • You cannot reply to topics in this forum
  • You cannot edit your posts in this forum
  • You cannot delete your posts in this forum
  • You cannot post attachments in this forum
 
 

© 1998-2014. Ozzu® is a registered trademark of Unmelted, LLC.