Posts Tagged ‘mysql’

Codeigniter ActiveRecord stupidity

Monday, July 14th, 2008

While working with CI is moderately fun, sometimes this framework is getting on my nerves. The ActiveRecord, apart from the fact that it has nothing to do with ROR ActiveRecord philosophy is pretty annoying when it comes to “more complex” (not awfully dumb) queries. Let’s assume the following query (plain sql)

SELECT product.id AS id, name, shortdesc, price, hidden, issmall,
uniqueness, images, COUNT(product_id) AS categorynum
FROM product
LEFT JOIN productcategory AS pc
ON product.id = pc.product_id
GROUP BY product_id
ORDER BY name ASC
LIMIT 15
OFFSET 2

This is not that complicated, is it? Product and category is a many to many relationship via productcategory, everything else should be pretty straightforward, including the pagination. Btw offset is a valid keyword, the Jedit hiliter is a bit outdated.

Since I don’t want to spend my time rewriting this query to something activerecordish, but while I still am in need of escaping/filtering, I would do something like this (which would not work):

$sql = '
SELECT product.id AS id, name, shortdesc, price, hidden, issmall,
uniqueness, images, COUNT(product_id) AS categorynum
FROM product
LEFT JOIN productcategory AS pc
ON product.id = pc.product_id
GROUP BY product_id
ORDER BY ? ?
LIMIT ?
OFFSET ?
';
$query = $this->db->query($sql, array($sort, $dir, $perpage, $offset));

The problem is the order by clause, which does have parameters from outside php, so should be filtered, but is enclosed with backticks (yes, I can turn off escaping, but that’s not what I want to, especially not for all the fields). By the way, the binding is done with question marks, which is a common practice, but which is in my opinion can be very annoying. As a sidenote, I think the “%VAR% -> foreach replace with array[VAR]” template-like syntax is much better (for quick and simple solutions), but back to the original topic, moving onto the final solution:

$this->db->select("product.id AS id, name, shortdesc, price, hidden, 
issmall, uniqueness, images, COUNT(product_id) AS categorynum");
$this->db->join('productcategory AS pc', 'product.id = pc.product_id');
$this->db->group_by('product_id');
$this->db->order_by($sort, $dir);
$this->db->limit($perpage, $offset);
$query = $this->db->get('product');

Is this nicer than the original? For hardcore php fans it may be, but to me it is pretty much the same as the original sql code; add the fact that I’m stuck with this solution, I tend to dislike it even more than hybrid sql queries (from different orm layers).

Select best rows of a group, mysql madness starts

Sunday, February 24th, 2008

At first, if someone’s not very familiar with sql (yes, like myself) may think, that grouping results in a table will automatically do the grouping on the required rows. Here’s an example table (no, it is not normalized, this is a dumb example only):

id   game                     score  developer
------------------------------------------------
1 Secret of Mana 8 Square
2 Final Fantasy VII 7 Square
3 Maniac Mansion 8 Lucasarts
4 Treasure Hunter G 6 Square
5 Star Ocean Ex 9 Square
6 Secret of Monkey Island 9 Lucasarts

Let’s say, we want to know the best game of each developer:

SELECT developer, max(score), game
FROM games
GROUP BY developer

The result is:
3, Maniac Mansion, 9, Lucasarts
1, Secret of Mana, 9, Square

This clearly is WRONG. It is not rows that are grouped of course, so one has to recycle this query with something similar:

SELECT games.*
FROM (
SELECT developer, MAX(score) AS maxscore
FROM games
GROUP BY developer
) AS t1
INNER JOIN games
ON (t1.developer = games.developer AND t1.maxscore = games.score)

The result is:
5, Star Ocean Ex, 9, Square
6, Secret of Monkey Island, 9, Lucasarts

Now, oh my, this is a derived table and is not very nice – at least it’s performance wise, but still is ugly. Especially since tomorrow I have to do something like this with an extra (secondary) grouping. In case I haven’t mentioned before, this is a PHP-Apache-MySQL combo – but back to the query itself, it seems to be natural, that somehow I want to make this query a bit nicer.

One would want to create a view for this query, so that it can be reused for a second grouping, but upon fabricating the query we have a slight problem:

Error Code : 1349
View’s SELECT contains a subquery in the FROM clause

Which is okay, since that’s what the manual says. Or, I’d rather put it this way: it’s not okay, but that’s it, no subqueries for Deadcabbit today. Another possibility is breaking up the queries into subsequent views, but that can lead to performance issues eventually.

The other possibility is to use temporary tables, but this gives us more headaches than advantages:

Still I could use a stored procedure, but eventually that would boil down to the question, where do I want to store most of the db logic? While I think this may be the go for me (have I mentioned before I’m no db expert?), right now I’ll keep things on the php part.

I would be happy to see some constructive comments for this post, but usually it’s only me who reads this journal ;) BTW Xaprb’s blog seems to be really helpful with all things mysql.

Mysql utf-8 data retrieveal from db

Tuesday, February 19th, 2008

This really gave me a headache: both on Linux and on Windows, php5, Apache and Mysql no matter how hard I tried, mysql gave back latin-1 encoded characters. To make a long story short:

$link=mysql_connect(‘localhost’, ‘root’, ‘root’);
mysql_query(“SET CHARACTER SET ‘utf8′”, $link);

saved the day. I think it happened sometime after MySql 4.0; all files saved in utf, my.cnf, httpd.conf edited and all the table collation was not enough.

Hibernate vs. complex database logic

Saturday, December 1st, 2007

We had been talking about Hibernate and how our whole db model is crippled, so I thought of doing some research on the topic – it seems to me, that I’m not the first to do so:

Read all the comments, they worth it. Now I understand things are not black and white here either :) Some thoughts:

  • HQL is not evil, sometimes it really is needed
  • HQL and the whole ORM paradigm is database-agnostic; high level SQL (mostly sps) is not
  • really pure db logic is huge and risomatic, hard to maintain and tend to be way too complex
  • code generators and code aids are important (see MiddleGen) and can be very useful
  • add to the above comment that while ROR does most of the magic automatically POJO/Hibernate Objects can be easier to tune
  • iBatis and Hibernate are tools, they can be used for (somewhat) different things
  • with legacy db code it is easier to add an ORM layer than to employ db backend logic
  • and many other things :)

InnoDB vs MyIsam

Tuesday, May 15th, 2007

Okay, I started goofing around in my freetime (read: approx one hour a day, when I’m not overworking like mad, which happens way too much nowdays) once again with world’s dirtiest scripting language; sitting on a LAMP system seemed to be a good idea to look at some of the code I wrote long ago – maybe I can come up with better ideas.

I have this database, MyIsam tables for fast fulltext searching, but hell, I’d like to have foreign keys! :) Either or, damn you MySQL. And besides, MySQL fulltext search is pretty much crap, so I either implement a search system for myself (stored procedures + fulltext replication from InnoDb to MyIsam + custom hit ranking) or I start looking into Zend’s search (read: php Lucene)… neither one seems to be painless. If I had the time, I would go for Ruby of course :(