Monday, March 22, 2010

CF9 and Amazon S3 - Has anyone got this...

I've now sorted my other issues and thanks to everyone who helped me with them - I've now moved on to trying to create a set of services that will connect to the S3 cloud and post and retrieve pdf files in our buckets - We need to be able to supply vast amounts of files to client and the cost of that currently is exorbitant in server disk space compared to with using the cloud

I've searched all over the net and picked up a number of ways of doing this - None of which seem to be able to generate the correct signature- I'm currently using the cf_hmac stuff from Adobe

This is my code

%26lt;cfobject component=''AmazonWebServices.amazons3'' name=''amazonS3''%26gt;
%26lt;cfset s3ServiceKeys = structNew()/%26gt;
%26lt;cfset s3ServiceKeys.accessKey = ''%26lt;MyKey%26gt;''/%26gt;
%26lt;cfset s3ServiceKeys.secret = ''%26lt;MySecretKey%26gt;''/%26gt;
%26lt;cfset s3ServiceKeys.bucket = ''%26lt;MyBucketName%26gt;''/%26gt;
%26lt;cfset s3serviceKeys.pdfContentType = ''application/pdf''/%26gt;
%26lt;cfset s3serviceKeys.gifContentType = ''image/gif''/%26gt;
%26lt;cfset s3serviceKeys.jpegContentType = ''image/jpeg''/%26gt;
%26lt;cfset s3serviceKeys.requestType = ''cname''/%26gt;

%26lt;cfset deliverDir = ''%26lt;MyDeliverDir%26gt;''/%26gt;

%26lt;cfset s3Service = amazonS3.init(awsKey=''#s3ServiceKeys.accessKey#'',awsSecret=''#s3ServiceKeys.secr et#'')/%26gt;

%26lt;cfdirectory action=''list'' directory=''#deliverDir#'' name=''TheFileList'' sort=''dateLastModified'' recurse=''no'' type=''file''%26gt;

%26lt;cfoutput query=''TheFileList''%26gt;
%26lt;cfset result = s3Service.putFileOnS3(localFilePath=''#deliverDir#\#TheFileList.name#'',
聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽 contentType=''#s3serviceKeys.pdfContentType#'',
聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽 requestType=''#s3serviceKeys.requestType#'',
聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽 bucket=''#s3serviceKeys.bucket#'',
聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽 objectKey=''#TheFileList.name#'')/%26gt;
%26lt;cfdump var=''#result#''/%26gt;
%26lt;/cfoutput%26gt;

This is the AmazonS3Service - more or less a direct copy from here http://www.barneyb.com/barneyblog/projects/amazon-s3-cfc/

%26lt;cffunction name=''init'' access=''public'' output=''false'' returntype=''amazons3''%26gt;
聽聽聽聽聽聽聽聽 %26lt;cfargument name=''awsKey'' type=''string'' required=''true'' /%26gt;
聽聽聽聽聽聽聽聽 %26lt;cfargument name=''awsSecret'' type=''string'' required=''true'' /%26gt;
聽聽聽聽聽聽聽聽 %26lt;cfargument name=''localCacheDir'' type=''string'' required=''false''
聽聽聽聽聽聽聽聽聽聽聽聽 hint=''If omitted, no local caching is done.聽 If provided, this directory is used for local caching of S3 assets.聽 Note that if local caching is enabled, this CFC assumes it is the only entity managing the S3 storage and therefore that S3 never needs to be checked for updates (other than those made though this CFC).聽 If you update S3 via other means, you cannot safely use the local cache.'' /%26gt;
聽聽聽聽聽聽聽聽 %26lt;cfset variables.awsKey = awsKey /%26gt;
聽聽聽聽聽聽聽聽 %26lt;cfset variables.awsSecret = awsSecret /%26gt;
聽聽聽聽聽聽聽聽 %26lt;cfset variables.useLocalCache = structKeyExists(arguments, ''localCacheDir'') /%26gt;
聽聽聽聽聽聽聽聽 %26lt;cfif useLocalCache%26gt;
聽聽聽聽聽聽聽聽聽聽聽聽 %26lt;cfset variables.localCacheDir = localCacheDir /%26gt;
聽聽聽聽聽聽聽聽聽聽聽聽 %26lt;cfif NOT directoryExists(localCacheDir)%26gt;
聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽 %26lt;cfdirectory action=''create''
聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽聽 directory=''#localCacheDir#'' /%26gt;
聽聽聽聽聽聽聽聽聽聽聽聽 %26lt;/cfif%26gt;
聽聽聽聽聽聽聽聽 %26lt;/cfif%26gt;
聽聽聽聽聽聽聽聽 %26lt;cfreturn this /%26gt;
聽聽聽聽 %26lt;/cffunction%26gt;

%26lt;cffunction name=''putFileOnS3'' access=''public'' output=''false'' returntype=''struct''
聽聽聽聽聽聽聽聽聽聽聽 hint=''I put a file on S3, and return the HTTP response from the PUT''%26gt;
聽聽聽聽聽聽聽 %26lt;cfargument name=''localFilePath'' type=''string'' required=''true'' /%26gt;
聽聽聽聽聽聽聽 %26lt;cfargument name=''contentType'' type=''string'' required=''true''聽 /%26gt;
聽聽聽聽聽聽聽 %26lt;cfargument name=''bucket'' type=''string'' required=''true'' /%26gt;
聽聽聽聽聽聽聽 %26lt;cfargument name=''objectKey'' type=''string'' required=''true'' /%26gt;
聽聽聽聽聽聽聽 %26lt;cfset var gmtNow = dateAdd(''s'', getTimeZoneInfo().utcTotalOffset, now()) /%26gt;
聽聽聽聽聽聽聽 %26lt;cfset var dateValue = dateFormat(gmtNow, ''ddd, dd mmm yyyy'') %26amp; '' '' %26amp; timeFormat(gmtNow, ''HH:mm:ss'') %26amp; '' GMT'' /%26gt;
聽聽聽聽聽聽聽 %26lt;cfset var signature = getRequestSignature(
聽聽聽聽聽聽聽聽聽聽聽 ''PUT'',
聽聽聽聽聽聽聽聽聽聽聽 bucket,
聽聽聽聽聽聽聽聽聽聽聽 objectKey,
聽聽聽聽聽聽聽聽聽聽聽 dateValue,
聽聽聽聽聽聽聽聽聽聽聽 contentType
聽聽聽聽聽聽聽 ) /%26gt;
聽聽聽聽聽聽聽 %26lt;cfset var content = '''' /%26gt;
聽聽聽聽聽聽聽 %26lt;cfset var result = '''' /%26gt;
聽聽聽聽聽聽聽 %26lt;cffile action=''readbinary''
聽聽聽聽聽聽聽聽聽聽聽 file=''#localFilePath#''
聽聽聽聽聽聽聽聽聽聽聽 variable=''content'' /%26gt;
聽聽聽聽聽聽聽 %26lt;cfhttp url=''http://s3.amazonaws.com/#bucket#/#objectKey#''
聽聽聽聽聽聽聽聽聽聽聽 method=''PUT''
聽聽聽聽聽聽聽聽聽聽聽 result=''result''%26gt;
聽聽聽聽聽聽聽聽聽聽聽 %26lt;cfhttpparam type=''header'' name=''Date'' value=''#dateValue#'' /%26gt;
聽聽聽聽聽聽聽聽聽聽聽 %26lt;cfhttpparam type=''header'' name=''Authorization'' value=''AWS #variables.awsKey#:#signature#'' /%26gt;
聽聽聽聽聽聽聽聽聽聽聽 %26lt;cfhttpparam type=''header'' name=''Content-Type'' value=''#contentType#'' /%26gt;
聽聽聽聽聽聽聽聽聽聽聽 %26lt;cfhttpparam type=''header'' name=''x-amz-acl'' value=''public-read''%26gt;
聽聽聽聽聽聽聽聽聽聽聽 %26lt;cfhttpparam type=''body'' value=''#content#'' /%26gt;
聽聽聽聽聽聽聽 %26lt;/cfhttp%26gt;
聽聽聽聽聽聽聽 %26lt;cfset deleteCacheFor(bucket, objectKey) /%26gt;
聽聽聽聽聽聽聽 %26lt;cfreturn result /%26gt;
聽聽聽 %26lt;/cffunction%26gt;

%26lt;cffunction name=''getRequestSignature'' access=''private'' output=''false'' returntype=''string''%26gt;
聽聽聽聽聽聽聽 %26lt;cfargument name=''verb'' type=''string'' required=''true'' /%26gt;
聽聽聽聽聽聽聽 %26lt;cfargument name=''bucket'' type=''string'' required=''true'' /%26gt;
聽聽聽聽聽聽聽 %26lt;cfargument name=''objectKey'' type=''string'' required=''true'' /%26gt;
聽聽聽聽聽聽聽 %26lt;cfargument name=''dateOrExpiration'' type=''string'' required=''true'' /%26gt;
聽聽聽聽聽聽聽 %26lt;cfargument name=''contentType'' type=''string'' default='''' /%26gt;聽聽聽聽聽聽聽
聽聽聽聽聽聽聽
聽聽聽聽聽聽聽 %26lt;cfset stringToSign = ''#trim(ucase(verb))#\n#contentType#\n#dateOrExpiration#\n/#bucket#/#objectKey#'' /%26gt;
聽聽聽聽聽聽聽聽聽聽聽
聽聽聽聽聽聽聽聽聽聽聽 %26lt;!--- Replace ''\n'' with ''chr(10) to get a correct digest ---%26gt;
聽聽聽聽聽聽聽聽聽聽聽 %26lt;cfset fixedData = replace(stringToSign,''\n'',''#chr(10)#'',''all'')%26gt;

聽聽聽聽聽聽聽聽聽聽聽 %26lt;!--- Calculate the hash of the information ---%26gt;
聽聽聽聽聽聽聽聽聽聽聽 %26lt;cf_hmac hash_function=''sha1'' data=''#fixedData#'' key=''#variables.awsSecret#''%26gt;
聽聽聽聽聽聽聽聽聽聽聽
聽聽聽聽聽聽聽聽聽聽聽 %26lt;!--- fix the returned data to be a proper signature ---%26gt;
聽聽聽聽聽聽聽聽聽聽聽 %26lt;cfset signature = ToBase64(binaryDecode(digest,''hex''))%26gt;
聽聽聽聽聽聽聽 %26lt;cfreturn signature%26gt;
聽聽聽 %26lt;/cffunction%26gt;

I don't appear to have any problems with what is being passed in - just the signature that is being created - All the code I've found is for earlier than CF9 and I'm therefore wondering if something has changed - although its more likely that I'm doing something I shouldn't be

I need to be able to get stuff up there programatically before I start creating URL's so - this is a big first step to overcome

CF9 and Amazon S3 - Has anyone got this...

Got to be honest this is driving me nuts - I've seen it working using a java testing suite - works fine so why isn't the hmac_sha1 encode working ? I've also now tried it on CF8 and get the same error - So its not Coldfusion Server per se that is the problem - its something in the code I'm doing wrong - it must be

CF9 and Amazon S3 - Has anyone got this...

I've actually found a work around for this problem - I took the Amazon S3 Library for Rest in Java http://developer.amazonwebservices.com/connect/entry.jspa?externalID=132%26amp;categor yID=47 and compiled the files below com into a jar - Which was dropped into the coldfusion server/lib

I then used the examples to build the following coldfusion function

%26lt;cffunction name=''putFileOnS3'' access=''public'' returntype=''any''%26gt;
聽聽聽聽聽聽聽 %26lt;cfargument name=''localFilePath'' type=''string'' required=''true'' /%26gt;
聽聽聽聽聽聽聽 %26lt;cfargument name=''contentType'' type=''string'' required=''true''聽 /%26gt;
聽聽聽聽聽聽聽 %26lt;cfargument name=''bucket'' type=''string'' required=''true'' /%26gt;
聽聽聽聽聽聽聽 %26lt;cfargument name=''objectKey'' type=''string'' required=''true'' /%26gt;聽聽
聽聽聽聽聽聽聽 %26lt;cffile action=''read''
聽聽聽聽聽聽聽聽聽聽聽 file=''#localFilePath#''
聽聽聽聽聽聽聽聽聽聽聽 variable=''content'' /%26gt;聽聽
聽聽聽聽聽聽聽 %26lt;cfscript%26gt;
聽聽聽聽聽聽聽聽聽聽聽 s3conn = CreateObject(''java'', ''com.amazon.s3.AWSAuthConnection'');
聽聽聽聽聽聽聽聽聽聽聽 s3conn.init(''#variables.awsKey#'',''#variables.awsSecret#'');
聽聽聽聽聽聽聽聽聽聽聽 s3object = CreateObject(''java'', ''com.amazon.s3.S3Object'');
聽聽聽聽聽聽聽聽聽聽聽 s3object.init(content.GetBytes());
聽聽聽聽聽聽聽聽聽聽聽 headers = createObject(''java'',''java.util.TreeMap'');
聽聽聽聽聽聽聽聽聽聽聽 contentTypeArray[1] = ''#contentType#'';
聽聽聽聽聽聽聽聽聽聽聽 headers.put(''Content-Type'',contentTypeArray);
聽聽聽聽聽聽聽聽聽聽聽 response = s3conn.put(bucket, objectKey, s3object, headers).connection.getResponseMessage();
聽聽聽聽聽聽聽聽聽聽聽 return response;
聽聽聽聽聽聽聽 %26lt;/cfscript%26gt;聽聽
聽聽聽 %26lt;/cffunction%26gt;

Which works perfectly - At last - I've just got to build the rest of the suite now so that we can upload, delete, create URLs etc

Very relieved I found a work around and part of me is very pleased that good old Java provided the solution having spent 9 years as a Java programmer before starting work in Coldfusion

No comments:

Post a Comment