Mark Drew started a small test comparing the different engines in terms of performance. Since some of the engines did not finish as good as expected, the comments on the blog entry reflected that atomar tests are not so significant and a real field test should be done.
This was something that Vince Bonfanti proposed and done this week. The results can be read in Vince's blog. A couple of people wanted Railo to be compared to the other engines as well. So I used the same code Vince used in order to test the performance of Railo in comparison to the other engines.
As I mentioned in a comment on the blog, I was surprised how good Bluedragon 7 competes with components since BD7 is still an interpreter. Nevertheless I added some other tests just to show the differences.
The machine I used was nothing special. It's my development laptop, a Sony Vailo VGN A417M with 1.5GB of RAM and every engine using 512MB of JVM size. Each test was recorded in the same way using Microsofts Stress Testing Tool with 3 simultaneous requests.
Since BD7 Beta II did so good with components, I added some tests with Structs. The first one just fills a large struct and the second one only creates many small structs.
With this second test you can see one of the main weaknesses of CFMX. Large structs should be avoided as often as possible. Railo and even BD are much faster the larger the structs grow.
Test 1 was the exact same Vince used:
<cfparam name="url.x" default="50">
<cfset x=url.x>
<cfloop from="1" to="#x#" index="i"><!---
---><cfset oItem = CreateObject('component', 'Person')><!---
---><cfset oItem.setname("Bob" & i )><!---
---><cfset oItem.setage(20)><!---
---></cfloop>
done!And the component code:
<cfcomponent>
<cfset this.name = "">
<cfset this.age = 0>
<cffunction name="setName">
<cfargument name="name">
<cfset this.name = arguments.name>
</cffunction>
<cffunction name="setAge">
<cfargument name="age">
<cfset this.age = arguments.age>
</cffunction>
</cfcomponent>
Here are the results:
Since I knew how good BD7 acts with components I tried the same with structs:
<cfparam name="url.x" default="50">
<cfset x=url.x>
<cfset stItem = StructNew()>
<cfloop from="1" to="#x#" index="i"><!---
---><cfset stItem[i].Bob= i><!---
---><cfset stItem[i].age = 20><!---
---></cfloop>
done!
Here are the results:
Now, since I know that CFMX does handles large strucs badly I added this test:
<cfparam name="url.x" default="50">
<cfset x=url.x>
<cfloop from="1" to="#x#" index="i"><!---
---><cfset stItem = StructNew()><!---
---><cfset stItem.Bob= i><!---
---><cfset stItem.age = 20><!---
---></cfloop>
done!
And the final results look like this:
But I must also say, that everyone should do the tests for himself in order to find out, if the designated CFML engine suits him.
13 responses so far ↓
1 Mark Drew // Dec 14, 2006 at 9:05 PM
Great news that you guys are also blogging!
2 Gert Franz // Dec 14, 2006 at 9:24 PM
You know. Don't trust a statistic you have not faked yourself :-)
Gert
3 Jeff Gladnick // Dec 14, 2006 at 9:56 PM
4 Peter J. Farrell // Dec 15, 2006 at 10:30 AM
Have you seen this post about CF MX's compiler:
http://www.nomachetejuggling.com/2006/12/07/coldfusions-compiler/
Since it has to do with loops, I wonder if you are attributing the slowness to structs etc to structs -- maybe it's the loop. Have you tried writing it in cfscript?
5 Gert Franz // Dec 15, 2006 at 10:38 AM
But as evryone else said and as I repeat again. This is not a field test. It would be much more trustworthy, if some independent institute or person would do the testing with a real application.
6 John Farrar // Dec 18, 2006 at 3:05 PM
Ralio have the ability to actually create implicit
setters and getters on CFCs. The other two have
property-less objects which does great by changing
how we write code and passing all property handling
into the methods to deal with protected scope (private)
properties and business rules on setters.
7 Gert Franz // Jan 15, 2007 at 11:59 AM
Gert
8 Mark Drew // Jan 15, 2007 at 2:56 PM
9 Gert Franz // Jan 15, 2007 at 4:09 PM
you can find it in our Customer Center. Just log in and go to the Beta 1.1 Page inside there.
http://www.railo.ch/en/index.cfm?treeID=167
Gert
10 garment wholesale // Sep 28, 2007 at 12:41 PM
11 Clomid // May 19, 2008 at 4:44 PM
setters and getters on CFCs.
12 Caleb Cushing (xenoterracide) // Sep 8, 2008 at 5:57 AM
13 Gert Franz // Sep 8, 2008 at 8:50 AM
we changed our point of view regarding performance a little bit. We think publishing these tests ourselves is not good. Since everyone could think this is faked or not accurate. I'd rather encourage others to do performance tests with Railo and other engines or scripting languages. This is then much more trustworthy. We definitely could do some tests against PHP, Ruby, Perl, but a question remains: What code to use. We could try to build a similar one, but again I'd rather have someone else do these tests and publish the results. If anyone does, just send us the link and we will place it somewhere on our website.
Leave a Comment