Lot of Notes

I have been working on designing solutions to some new customer requirements recently. Some requirements are written based on the existence of some notes on an artifact. I guess that is the only way they could specify the types of artifacts that need certain processing. Okay.

My solution was to run a query to find out if those notes existed. I did not have the exact text of the notes. But I had a bunch of examples. I figured I could do a query where the note text was LIKE some pattern I could match. My guess was that I could throw the query over to our DBAs, and they would tell me whether the performance would be acceptable.

A colleague was working on some other new requirements. He had the same notes to search for. His solution was to just read all the notes into memory or some collection. They he would implement the search for the patterns he needs. Now this might be a way to get the job done fast. I prefer to let Oracle handle the searches. We shall see which method has the superior performance.

Baby Steps

A friend of mine had some trouble transitioning from her community college to a four year one. Right now she is in limbo. Looks like she is reading books and playing video games. That is fun for a while. But I think she should be working on personal development projects.

I decided to work on some side projects with my friend to motivate her to start writing code again. She had an idea for a fiction database. She tried to do it once. But her queries were not working and she gave up. That is so not like her.

My first idea was for us to produce a minimum viable product (MVP) for her fiction database. I helped trim the specs down to a bare bones app. I cooked up a simple single database table to hold the data. Then I reviewed the SQL needed to get records of interest from that table. That's when I saw my friend had some trouble constructing the WHERE clause.

With the SQL confusion resolved, my friend banged out the app. I was happy. We decided to write the app in PHP and use MySQL on the back end. Next step is to modify the app to work with her more complicated database schema. We have the basics down. Moving forward should be easy.

ODP.NET

Just read a article that describes how to use the latest Oracle Data Provider for .NET, also known as "ODP.NET". You must know that there are type types of this provider. One is the managed driver. The other is the unmanaged one. I was interested in the unmanaged driver.

You need to have, at a minimum, Visual Studio 2010. At work I run with Visual Studio 2008. But at home I use Visual Studio 2010 for school. So I am good to go to try things out. Apparently you also need to download Oracle Developer Tools for Visual Studio, also know as ODT.

The example I read had me setting up my Visual Studio project to add a reference to Oracle DataAccess. Then in my code I would add using statements to Oracle.DataAccess.Client and Types. After that I should be good to go to use the driver.

There is some bolierplate for the data access code I write. Create an OracleConnection. Set the ConnectionString. Call the CreateCommand. Set CommandText to my SQL. Then ExecuteReader and Read. Simple right?

Performance Tradeoff

I remember the first huge PL/SQL stored procedure I wrote. The thing was highly modular. It came out to maybe 10k lines of code. I liked that no procedure or function was too long. I broke things down into small pieces. Did a lot of testing to ensure the functionality was correct. Then I shipped it to a production environment.

 

Immediately I got complaints that my stored procedure took too long to complete. The thing might have been running for almost an hour. Yeah. The data I was processing was on the order of millions of rows. I had to do a lot of crunching to determine the outputs I had to produce. Nobody told me about the performance requirements.

 

I received some help from some performance tuners. They had some tricks to make my jobs run in parallel. They rewrote some of my long running queries. Some problems such as many SQL statements to ensure modularity could not be fixed without a rewrite. Ouch. In the end, I think the average time of my runs was brought down to 20 minutes. It was acceptable but not great.

 

Now I try to produce code that runs fast. From the get go, I code in a way that does not duplicate SQL. I try to get everything done in minimal large SQL select statements. However there comes a time when I might need to duplicate a SQL query to ensure the code base is maintainable. It is a delicate balance.

Errors in Report

A tester informed me with a number of problems with one of the reports out code produces. One specific problem was that the title of the report indicated there were errors. The report itself was not that big. The SQL was a bit complex.

I asked for some logs from the server. When I poured through them, I found references to an ORA-14551. That error indicates some code is trying to do some DML in the middle of my SELECT statement. I did not see any explicit UPDATE/INSERT/DELETE statements.

Then I went to the end of the procedures called by the report. Turns out the exception handler tried to add some records to the database. Ooops. You can't do that in the middle of a SELECT statement. I think I could just push that error handling down to the report, out of the stored procedure. Problem solved.