I've got a hot new side project. There is a list of 170 million URLs I want to scrape from the web. I want to use Oracle to hold my data. What version of the database do I need? My first choice was Oracle Express Edition, also known as Oracle XE. It is free. However it has a limit of 4 Gigabytes of user data. My first database table with 170 million records will blow past that limit.Next I took a quick look at the different Oracle Standard Edition offerings. They go from $180 to $350 per user, with a 5 user minimum. This cost is not astronomical. But my project is in a research mode right now. I don't want t0 shell out that kind of cash unless I have a money making opportunity. So I will do a prototype with Oracle XE.
Oracle XE comes with a Run SQL Command Line tool. It looks like a DOS box, and behaves like SQL*Plus. That is too bare bones for me. Let's see what kind of tools there are out there that can help me. I am used to PL/SQL Developer myself. However others swear by Toad. And Oracle has SQL Developer. Next time I will let you know my initial experience with these tools for my new project.
