S3 with Railo

March 31, 2008 · By Gert Franz · 17 Comments

Some of you may know that there is a Amazon S3 CFC you can download from RiaForge. Railo has an own way of handling Amazon S3 resources.When you want to use Amazon S3 it can become quite challenging to use the service for storing and reading data with CFML. Here the amazon S3 CFC helps a little.
Since Railo 2.0 introduced resources (virtual file systems) and everything file based in Railo is based on these resources, we have implemented a new resource called S3. This resource can be used as any other resource as well. So if you have an accessKeyID and a secretAccessKey, you can use it quite easy like follows:
<cffile action="copy" source="s3://[accessKeyID]:[secretAccessKey]@s3.amazonaws.com/bucketname/susi.txt" destination="c:\susi.txt">
Now, if you define a mapping that points to a S3 resource like follows:

MappingName:
/myS3Storage
Resource path
s3://[accessKeyID]:[secretAccessKey]@s3.amazonaws.com/bucketname/

you can use it in your code like this:
<cfinclude template="/myS3Resource/susi.cfm"> <!--- or --->
<cfif fileExists("/myS3Resource/susi.cfm")>
...
</cfif>
<cfdirectory dir="/myS3Resource" action="list" name="s3Files>
<cfdump eval="s3Files">

Please note that if you don't define the bucket name you can list all your buckets by using CFDIRECTORY on it. So for example it is very easy to store your video files you convert with to the Amazon S3.

Updated: December 15th, 2009

Railo's Amazon S3 extension is a paid extension. If you need information about availability / cost, please Contact us.

Updated: December 15th, 2010

The Amazon S3 functionality is now baked into the Railo core and available in the Railo 3.2 release free of charge.

View count: 2177

Tags: Features · Railo 2.0

17 responses so far ↓

  • 1 Jeff Gladnick // Mar 31, 2008 at 4:46 AM

    That is quite excellent - any plans to release an amazon ec2 image with Railo/Linux/Apache/MySQL installed?

  • 2 David Low // Mar 31, 2008 at 11:59 AM

    This is excellent news... I remember asking about S3 resources this time last year and it's going to be hugely helpful for a few things... cheers Gert.

  • 3 jeff // Mar 31, 2008 at 4:57 PM

    This is a really cool idea.
    We have started using S3 for lots of things and this would make dealing with S3 easier.

    Although the s3 cfc is really great.

  • 4 Joe Potenza // Mar 31, 2008 at 10:33 PM

    I have a server with Railo community 2.0.0.026 final installed. I've tried setting up a mapping exactly as you've described, and also by addressing it directly and it keeps giving me an error such as &quot;/myS3Storage doesn't exist&quot; or &quot;directory [/path/to/railo/s3:/accesskey:secretkey@s3.amazonaws.com] doesn't exist&quot; (that second error shows &quot;s3:/&quot; even though I address it as &quot;s3://&quot;)

    Do I need a newer version of Railo, or to install anything additional? I already use the S3 cfc with success, but I'd like to test this method and see which works best for me.

  • 5 Gert Franz // Apr 1, 2008 at 8:04 AM

    Hi all,
    at the moment we are experimenting with a public amazon ec2 image with Railo/Linux/Apache/MySQL. As soon as we have it ready for playing around, I will write here about it.
    At Scotch I will demonstrate how to use the different types of resources, for any of you who want to know how this works.
    @Joe
    Joe, you need at least the Railo version 2.0.0.038 in order to play with the amazon s3 resource. After having updated to it, (at the moment) you have to manually tell Railo where to find the resource. This can be done in the railo-web.xml.cfm file. Just add the following entry to the &lt;resources&gt; element (as an example below, line 3):
    1: &lt;resources&gt;&lt;!-- arguments: lock-timeout - define how long a request wait for a log --&gt;
    2: &lt;resource-provider arguments=&quot;case-sensitive:true;lock-timeout:1000;&quot; class=&quot;railo.commons.io.res.type.ram.RamResourceProvider&quot; scheme=&quot;ram&quot;/&gt;
    3: &lt;resource-provider arguments=&quot;lock-timeout:10000;&quot; class=&quot;railo.commons.io.res.type.s3.S3ResourceProvider&quot; scheme=&quot;s3&quot;/&gt;
    4: &lt;/resources&gt;
    then you can use the Amazon s3 resource like I described. We will officially release the Amazon S3 resource with Railo 2.1.
    Let me know how it works.
    Gert

  • 6 Brad B. // Apr 1, 2008 at 9:09 AM

    This is really great, the problem is it will completely kill your web application's portability. Now that BlueDragon is scheduled to open it's source, things like compatibility become more important. I hope we start to focus a little more on standards in general as a community.

  • 7 Streit Michael // Apr 1, 2008 at 9:56 AM

    Compatibility is always a major concern for us, we are spending as much many time on compatibility as possible. But we can't be compatible with features that are not supported by other engines. Resources are a very powerfull tool, but if you want to write an application that must be compatible with other cfml engines, simple don't use it.

  • 8 Joe Potenza // Apr 1, 2008 at 10:38 AM

    Gert - thanks for the info. Where do I get a copy of 2.0.0.038? Your site shows the latest version as 2.0.0.034. What do I need to do to upgrade?

    Thanks,
    Joe

  • 9 Gert Franz // Apr 1, 2008 at 11:48 AM

    You can upgrade your local version by entering http://preview.railo.ch into the update path in the server administrator. Then hit the update button.

    Gert

  • 10 Mike Kelp // Apr 1, 2008 at 1:14 PM

    Great use of that feature!

    You guys are really innovating with CF. Awesome work.

    Mike.

  • 11 Joe Potenza // Apr 1, 2008 at 10:48 PM

    Works great, thanks! I'll be testing it over the next few days as far as performance goes.

    FYI, after updating my railo installation to 2.0.0.043, calls to facebook's API stopped working. I discovered that the problem was the new version seems to UCASE the key names when looping over a structure. Simply putting an LCASE() around the key name when passing it to facebook fixed the problem. Not sure if this will affect anybody else, but it's something to keep in mind.

    Joe

  • 12 Andrew Grosset // Jan 5, 2009 at 9:09 PM

    Works great! dont forget to remove the &quot;[ ]&quot; around your access keys,

    Andrew.

  • 13 Andrew Grosset // Jan 12, 2009 at 6:37 AM

    having trouble setting acl permissions in amazon s3, although the acl below is set to &quot;public-read&quot; it is in fact being set to &quot;private&quot; ie the acl setting is being ignored. I am using &quot;railo-3.0.1.000-railo-express-6.1.0-with-jre-windows&quot;

    &lt;cffile
    action = &quot;copy&quot;
    source = &quot;C:\mystuff\pics\balloons3d.jpg&quot;
    destination = &quot;/myS3Storage/andrew/files/balloons3d.jpg&quot;
    acl = &quot;public-read&quot;
    &gt;

    Andrew.

  • 14 Jeff // Apr 23, 2009 at 6:27 AM

    Having trouble trying to connect to AWS. I put in my key and secrety key just like the blog post described, but the Railo admin won't save it. Just highlights the resource in red.

    However, I am able to connect with a different freeware utility using the same credentials/

  • 15 Gert Franz // Apr 23, 2009 at 8:47 AM

    Jeff,

    are you using Railo 3.0 or 3.1? This is an extension now in Railo 3.1. In addition you might need to add a certain JAR file and restart the engine.
    Have a look at Mark's latest blog posts: http://www.markdrew.co.uk

    Gert

  • 16 Dave Anderson // May 23, 2009 at 6:04 AM

    I'm new to Railo and S3...where do 'buckets' come into play in all this? Should there not be a bucket name in that path? Are they treated as directories, and thus can be created with cfdirectory if there's an s3 mapping?

  • 17 Gert Franz // May 29, 2009 at 10:00 AM

    @dave:

    The buckets are just like directoriy names appended at the end of the s3resource. Sou your definition can look like this:

    s3://[accessKeyID]:[secretAccessKey]@s3.amazonaws.com/[bucketname]

    Gert

Leave a Comment

Leave this field empty: