Simon Whatley » web application

Canonical URLs – What Are They All About?

Simon — Wed, 07 Oct 2009 09:34:23 +0000

Google announced as long ago as February, in their official Webmaster Central Blog a new canonical URL tag:

Carpe diem on any duplicate content worries: we now support a format that allows you to publicly specify your preferred version of a URL. If your site has identical or vastly similar content that’s accessible through multiple URLs, this format provides you with more control over the URL returned in search results. It also helps to make sure that properties such as link popularity are consolidated to your preferred version.

But what do they mean by canonical? One of the definitions of canonical is reduced to the simplest and most significant form possible without loss of generality.

What this means is that if you have a page–let’s take an e-commerce product page–and the simplest URL that you want it accessible by is:

http://www.site.com/category/product.html

you can add the canonical tag to that specific product. Google, Yahoo and Microsoft use this tag to tell their search engines which URL it should have for the current page.

Now, let’s say that the particular software you use also allows you to access the same product using:

http://www.site.com/company/product.html

and

http://www.site.com/different_category/product.html

Perhaps this one product is in multiple categories. With this tag in place when any of the alternate pages are loaded this tag notifies any search engine that this is really the same product as the page you defined in the canonical tag. So, you are still allowed to have the content available as generally needed (by categories, tags, or some other organisation system) and still avoid having the content duplicated and penalised.

To implement the canonical URL tag in your web application, you simply need to do the following inside the section of the duplicate content URLs:

As Google mention, this tag is a hint that they honour strongly. Google will take your preference into account, in conjunction with other signals, when calculating the most relevant page to display in search results.

Secure Your Application – PCI DSS Specifications

Simon — Mon, 26 Jan 2009 11:19:37 +0000

PCI DSS stands for Payment Card Industry Data Security Standard, and is a worldwide security standard assembled by the Payment Card Industry Security Standards Council (PCI SSC). The PCI security standards are technical and operational requirements that were created to help organizations that process card payments prevent credit card fraud, hacking and various other security vulnerabilities and threats. The standards apply to all organizations that store, process or transmit cardholder data – with guidance for software developers and manufacturers of applications and devices used in those transactions. A company processing, storing, or transmitting cardholder data must be PCI DSS compliant.

The PCI SSC (Council) is responsible for managing the security standards, while compliance with the PCI set of standards is enforced by the founding members of the Council: American Express, Discover Financial Services, JCB International, MasterCard Worldwide and Visa Inc. Non-compliant companies who maintain a relationship with one or more of the card brands, either directly or through an acquirer risk losing their ability to process credit card payments and being audited and/or fined.

All in-scope companies must validate their compliance annually. This validation can be conducted by Qualified Security Assessors, i.e. companies that have completed a three-step certification process by the PCI SSC which recognises them as being qualified to assess compliance to the PCI DSS standard. However, smaller companies have the option to use a Self-Assessment Questionnaire. Whether this questionnaire needs to be validated by a QSA depends on the requirements of the card brands in that merchant’s region.

The current version of the standard specifies 12 requirements for compliance, organised into 6 logically related groups, which are called “control objectives.”

Build and Maintain a Secure Network
- Requirement 1: Install and maintain a firewall configuration to protect cardholder data
- Requirement 2: Do not use vendor-supplied defaults for system passwords and other security parameters
Protect Cardholder Data
- Requirement 3: Protect stored cardholder data
- Requirement 4: Encrypt transmission of cardholder data across open, public networks
Maintain a Vulnerability Management Program
- Requirement 5: Use and regularly update anti-virus software
- Requirement 6: Develop and maintain secure systems and applications
Implement Strong Access Control Measures
- Requirement 7: Restrict access to cardholder data by business need-to-know
- Requirement 8: Assign a unique ID to each person with computer access
- Requirement 9: Restrict physical access to cardholder data
Regularly Monitor and Test Networks
- Requirement 10: Track and monitor all access to network resources and cardholder data
- Requirement 11: Regularly test security systems and processes
Maintain an Information Security Policy
- Requirement 12: Maintain a policy that addresses information security

Compliance with these requirements can be summarized into 3 main stages:

Collecting and storing: Secure collection and tamper-proof storage of all log data so that it is available for analysis.
Reporting: Being able to prove compliance on the spot if audited and present evidence that controls are in place for protecting data.
Monitoring and alerting: Have systems in place such as auto-alerting, to help administrators constantly monitor access and usage of data. Administrators are warned of problems immediately and can rapidly address them. These systems should also extend to the log data itself –- there must be proof that log data is being collected and stored.

What does this actually mean for web application developers?

It is considerably more expensive and more time-consuming to recover from a security incident than to take preventative measures ahead of time. If you follow the guidelines below, you will go along way to securing you application in line with the PCI DSS regulations. Many of the measures apply to general application security, but since PCI DSS is all about security, they are worth mentioning.

Server-level Security:

Separate web- and database-servers on to different physical machines.
Secure the web- and database-servers with traditional techniques. Only authorised accounts should have the capabilities to run tasks on the machine. That means not giving admin-rights to the user account.
Keep servers up-to-date with the latest patches and software releases.
Minimise the number of services running on the server. This means limiting the services to only those required for the web- or database-servers to function.
Secure information in transit between servers. This may mean physically securing the network to prevent evesdropping via encryption or obfuscating the data amongst innocuous ‘noise’.
Secure the database server behind a firewall.

Application-level Security:

Separate ColdFusion, the webserver and database server user accounts. They should never be under the same system account.
Create a database user specifically for your ColdFusion datasource and restrict it to only the activities required for the application. The user should not have database-owner rights, access to databases not relating to the application or access to the system tables.
Revoke privileges in the ColdFusion datasource definition to prevent the SQL commands CREATE, DROP, GRANT, REVOKE and ALTER.
General settings in the ColdFusion Administrator:
- Check the Disable access to internal ColdFusion Java components option.
- Check the Enable Global Script Protection option.
- Add a Missing Template Handler.
- Add a Site-wide Error Handler.
- Reduce the Maximum size of post data from 100MB.
- Enable Timeout Requests, and set to 60 seconds or less.
- Disable Robust Exception Handling on production servers.

Web Application-level Security:

Use secure HTTP to transfer data and/or when logged into ‘administration’ secutions of your web application.
Timeout sessions after 15 minutes and on browser close.
Provide multi-level login processes. For example, lock the application after 3 failed attempts for a period of 10 minutes.
Do not identify whether the username or password are incorrect, simply notify the user that their login failed and that they must try again.
Encrypt passwords stored in the database with a standard such as SHA-256 or ‘stronger’.
Use CAPTCHAs (textual and aural) to prevent automated robots hacking into your application.
Run regular penetration tests on your application to identify potential problems.
Encrypt credit card information held in the database or other storage mechanism. Only store credit card data in line with the PCI DSS regulations.

Code-level Security:

Application.cfc – Set the scriptProtect Application variable to true to enable application-wide cross-site script protection.
CFQueryParam – This tag, importantly, verifies the data type of a query parameter and, for RDBMSs that support bind variables, enables ColdFusion to use bind variables in the SQL statement. Bind variable usage enhances performance when executing a cfquery statement multiple times. There are limitations to the use of the cfqueryparam tag. In ColdFusion 7 for example, you cannot use them in queries using the cachedWithin attribute. Similarly, they cannot be used in ORDER BY clauses, although the use of conditional logic should resolve the need for order by variables.
Functions – As a rule of thumb, validate all the data being passed into a query prior to it being used. ColdFusion MX 7 saw the introduction of the isValid() function. This function tests whether a value meets a validation or data type rule and can be used to replace a large number of type-specific functions such as isArray(), isBinary(), isBoolean(), isDate(), isNumeric() and isSimpleValue() etc.
Stored Procedures – I often favour the use of stored procedures over standard queries. Not only do they add an additional level of performance, they provide an additional level of security; ColdFusion does not do any raw processing of queries in the web code, it simply passes variables down the wire to the database server.

Conclusion

The goal of the PCI Data Security Standard is to protect cardholder data that is processed, stored or transmitted by merchants. The security controls and processes required by PCI DSS are vital for protecting cardholder account data, including the PAN – the primary account number printed on the front of a payment card. Merchants and any other service providers involved with payment card processing must never store sensitive authentication data after authorisation. This includes sensitive data that is printed on a card, or stored on a card’s magnetic stripe or chip – and personal identification numbers entered by the cardholder.

By following the points made above, you will go a long way to meeting the PCI DSS guidelines, whilst also securing your infrastructure and applications in a more general sense.

Caveat: The views and comments written in this article are provided as a guideline. I hold no responsibility for the security of your applications and data based upon the information provided.

Why Silverlight? Microsoft Explains…

Simon — Thu, 13 Dec 2007 22:12:00 +0000

Silverlight aims to compete with Adobe Flash and the presentation components of AJAX. It also competes with Sun Microsystems’ JavaFX, which was launched a few days after Silverlight.

Microsoft Silverlight is a proprietary runtime for browser-based Rich Internet Applications, providing a subset of the animation, vector graphics, and video playback capabilities of Windows Presentation Foundation. The runtime is available for Microsoft Windows and Mac OS X, with Linux support under development via the third-party Moonlight runtime.

Microsoft describes its advantages as follows:

Compelling Cross-Platform User Experiences

Deliver media experiences and rich interactive applications for the Web that incorporate video, animation, interactivity, and stunning user interfaces.
Seamless, fast installation for users, thanks to a small, on-demand, easy-to-install plug-in that is under 2 megabytes (MB) in size and works with all leading browsers.
Consistent experiences between Windows-based and Macintosh computers without any additional installation requirements.
Create richer, more compelling Web experiences that take greater advantage of the client for increased performance.
Stunning vector-based graphics, media, text, animation, and overlays that enable seamless integration of graphics and effects into any existing Web application.
Enhance existing standards/AJAX-based applications with richer graphics and media, and improve their performance and capabilities by using Silverlight.

Flexible Programming Model with Collaboration Tools

Based on the Microsoft .NET Framework, Silverlight enables developers and designers to easily use existing skills and tools to deliver media experiences and rich interactive applications for the Web.
Simple integration with existing Web technologies and assets means Silverlight works with any back-end Web environment or technology. No “rip and replace” required.
Silverlight integrates with your existing infrastructure and applications, including Apache, PHP, as well as JavaScript and XHTML on the client.
Choice of development languages including JavaScript, Ruby, Python, C#, Visual Basic .NET, and more.
Role-specific tools for both designers and developers that take advantage of Web standards and the breadth of the Microsoft .NET connected software features.
For designers: Microsoft Expression Studio for creating interactive user interfaces and media rich experiences, preparing media for encoding and distribution, and creating World Wide Web Consortium (W3C) standards-compliant sites using modern XHTML, XML, XSLT, CSS, and ASP.NET.
For developers: Microsoft Visual Studio for developing client and server code with full Microsoft IntelliSense, powerful cross-platform debugging, rich language support, and more.
Consistent presentation model by using XAML, the declarative presentation language used in Windows Vistaâ€“based applications. Controls, visual designs, media, and other elements can be presented with full design fidelity in both Silverlight and Windows-based applications.
Extensible control model makes it easy to add rich content and behaviors while enabling efficient code-reuse and sharing.
Dramatically improved performance for AJAX-enabled Web sites with the power, performance, and flexibility of Silverlight and .NET-connected software.

High Quality, Low Cost Media

Unified media format that scales from high definition (HD) to mobile with Windows Media Video (WMV), the Microsoft implementation of the Society of Motion Picture and Television Engineers (SMPTE) VC-1 video standard, as well as support for Windows Media Audio (WMA) and MP3 audio.
Add vector-based graphics and overlays to media with support for integration of graphics that scale to any size and broadcast-style overlays for tickers and closed captioning.
Flexible ad-insertion solutions with video and animation, including the ability to deliver fluid, broadcast-style video or animated advertisements without loss of visual fidelity or motion quality.
Lower-cost media streaming with Emmy Award winning Windows Media technologies that can lower the cost of streaming delivery by up to 46%, and enjoy the flexibility to work with your existing Windows Media streaming deployments. Even further cost reductions are possible with the upcoming Microsoft Internet Information Services (IIS) Media Pack for Microsoft Windows Server 2008.
Broad ecosystem of media tools, servers, and solutions compatible with the Windows Media operating system.
Microsoft PlayReady content-access technology that delivers a single solution for digital rights management support on both Windows-based and Macintosh computers for content providers (coming in Silverlight 1.1)
Powerful encoding tools for live and on-demand publishing of media experiences with Microsoft Expression Encoder, including hardware-accelerated encoding of WMV and VC-1 at up to 15 times the performance of software alone when paired with a Tarari Encoder Accelerator board.

Connected to Data, Servers, and Services

Mash-up and incorporate services and data from the Web by taking advantage of the Silverlight support for LINQ while accessing that data with common protocols like JSON, RSS, POX, and REST.
Increase discoverability of rich interactive application (RIA) content that can be indexed and searched due to the text-based XAML format that describes interface and content in a Silverlight-based application.
Rapidly scale applications with Silverlight Streaming by Windows Live to host and integrate software services and media content.

Streaming audio and video

Silverlight Streaming by Windows Live offers a free streaming and application hosting solution for delivering high-quality, cross-platform, cross-browser, media-enabled rich interactive applications (RIAs). With the ability to author content in Microsoft Expression Encoder and other third-party editing environments, Web designers maintain complete control of the user experience.

Eliminating Whitespace in ColdFusion

Simon — Fri, 26 Oct 2007 09:29:35 +0000

ColdFusion has never satisfactorily removed whitespace from generated content, however, removing this whitespace can dramatically improve your website’s performance. Take a 100KB page for example. If 20% of the page is made up of whitespace, that is 20% that is unneccessary and 20% bandwidth cost that can be saved.

Currently there are two main ways a ColdFusion developer can prevent whitespace; via the ColdFusion Administrator and by including certain tags in their code:

1. ColdFusion Administrator

Under the Server Settings > Settings menu item there is a checkbox called ‘Enable Whitespace Management’. This checkbox should always be checked. According to the description, checking this checkbox “reduces the file size of the pages that ColdFusion returns to the browser by removing many of the extra spaces, tabs, and carriage returns that ColdFusion might otherwise persist from the CFML source file.” I am yet to be convinced, but it is worthwhile enabling it.

2. ColdFusion Tags

The cfsetting tag controls aspects of page processing, such as the output of HTML code in pages (inside and outside the cfsetting tag body). If enablecfoutputonly is set to true, HTML output is blocked if it is not wrapped in a cfoutput tag. Therefore, this tag ‘can’ be used to minimise the amount of generated whitespace.

The cfsilent tag supresses output produced by CFML within a tag’s scope. Therefore, you simply need to wrap the tag around anything you don’t want to output to the browser. As it does not return anything from with in it, so you have to be a little be careful when debugging.

One of the cfprocessingdirective‘s purposes is to remove excess whitespace from ColdFusion generated content in the tag body. However, it does not affect any whitespace in HTML code. You can nest the tags and toggle supresswhitespace on and off, not that I ever identify a good reason to do so.

But there is a little known third way, using a servlet filter called Trim Filter.

Servlet filters are tools available to web application developers. They are designed to be able to manipulate a request and response (or both) that is sent to a web application.

Trim Filter lets you decrease the size of file your server will send to all clients. The filter removes extra spaces and line breaks from outputted documents. This can be especially useful for WAP/WML developers working with mobile devices, where the size for transferred documents is limited.

Setting up the servlet filter in ColdFusion

Using the Trim Filter servlet filter found at the following URL:

http://www.servletsuite.com/servlets/trimflt.htm

Download the trimflt.jar
Save trimflt.jar in the WEB-INF/lib folder in the ColdFusion Server wwwroot.
Edit the web.xml file in the WEB-INF folder in the ColdFusion Server wwwroot with the following code:


	trimFilter
	*.cfm

 

	trimFilter
	com.cj.trim.trimFilter

Restart the ColdFusion Service

And now, when a ColdFusion page is invoked, the whitespace is suppressed and wow does it do a good job.

Poor Man's HTTP Compression with ColdFusion

Simon — Sat, 17 Mar 2007 15:11:23 +0000

An interesting article I read in the CFDJ recently was entitled Poor Man’s HTTP Compression with ColdFusion. Almost every web application will benefit from the compression of content. A compression filter optimises the size of the content that is sent from a webserver to a web browser via the Internet. Since generating content and serving pages via the World Wide Web is the core behind web applications, it is simple components that aid these processes that are incredibly useful. This is where servlet filters come into play.

Servlet filters are tools available to web application developers. They are designed to be able to manipulate the request and responses that are sent to a web application, without manipulating the servlets, static pages like HTML and, in this case, CFM pages that are being used by the web application (unless of course that is the desired response). Servlet filters act like a chain of steps that a request and response must go through before reaching the page in the application.

Compressing Content Using a Servlet Filter

Compression is a process that reduces the number of bytes required to define a document in order to save disk space or transmission time. It is extremely useful for sending information across the web, because the speed at which people receive information from a web application is dependent upon how much data you are trying to send. The smaller the amount of information that is to be sent, the faster it can be sent. Therefore, compression and the associated responsiveness is a key component to retaining users and generating revenue from those retained users.

Compression can be effectively achieved by having a servlet filter conditionally pipe the produced content to a GZip-compressed file. GZip is supported by the HTTP protocol and almost all modern browsers (hence the servlet filter conditionally compresses the content).

GZip compression usually results in a 6:1 compression ratio, although this depends on how much content is being sent and what the content is.

Setting up the Servlet Filter in ColdFusion

Using the ColdBeans servlet filter found at the following URL:

http://www.servletsuite.com/servlets/gzipflt.htm

Download the GZipFilter.jar
Save GZipFilter.jar in the WEB-INF/lib folder in the ColdFusion Server wwwroot.
Edit the web.xml file in the WEB-INF folder in the ColdFusion Server wwwroot with the following code:
```
GzipFilter
com.cj.gzipflt.GzipFilter


GzipFilter
*.cfm
```
Restart the ColdFusion Service

Now, when you invoke any .cfm page the GzipFilter will check out client’s browser settings. If the browser does not support gzip, the filter invokes resource normally. If the browser does support gzip, output will be compressed.

ColdFusion & Lucene

Simon — Sun, 09 Apr 2006 22:54:09 +0000

One of the many reasons to use ColdFusion MX is that it has a large, standard toolset that enables the creation of full-featured, dynamic Web applications. The tag-based language makes it relatively simple to query a relational database and send e-mail. In a similar way, you can create and search Verity full-text indexes.

However, there are situations where you cannot use the full-text searching capabilities of Verity. For example, Verity only runs on Windows, Linux and Solaris, therefore the ability to run ColdFusion MX on the Apple OS X operating system, whilst advantageuos to developers who code on the Apple platform, does not include the ability to use Verity. Furthermore, programmers who work in a hybrid J2EE/ColdFusion MX environment cannot natively use the Verity search capabilities in the J2EE environment. Finally, programmers who need customized searching and indexing capabilities may find the standard Verity integration limiting. There are work-arounds include installing Verity on a Windows, Linux, or Solaris server and configuring your ColdFusion server to use the remote Verity server, however these may not only be impractical, but cost-prohibitative.

Enter Lucene, an open source full-text searching framework from the Apache Jakarta project, which, when combined with ColdFusion MX, can be run on Apple OS X, can be programmatically accessed by both J2EE and ColdFusion MX developers, and can be fully customized and extended.Apache Lucene is a high-performance, full-featured text search engine library written entirely in Java. It is a technology suitable for nearly any application that requires full-text search, especially cross-platform.

Apache Lucene is an open source project available for free download.

Features

Lucene offers powerful features through a simple API.

Scalable, High-Performance Indexing

Over 20MB/minute on Pentium M 1.5GHz
Small RAM requirements — only 1MB heap
Incremental indexing as fast as batch indexing
Index size roughly 20-30% the size of text indexed

Powerful, Accurate and Efficient Search Algorithms

Ranked searching — best results returned first
Many powerful query types: phrase queries, wildcard queries, proximity queries, range queries and more
Fielded searching (e.g., title, author, contents)
Date-range searching
Sorting by any field
Multiple-index searching with merged results
Allows simultaneous update and searching

ColdFusion & Lucene Implementations

If you don’t fancy attempting the task of writing your own ColdFusion implementation of Lucene, below are a couple of projects that will give you a kick-start along the road to indexing database content. With the addition of other Java projects such as PDFBox the textual content of a pdf can also be extracted and indexed.

Aaron Johnson

Inspired by Lindex, Aaron Johnson has created a CFX Tag called CFX_Lucene that closely mimics the ColdFusion cfsearch tag, but uses Lucene rather than Verity.

http://cephas.net/blog/lucene/index.html

CFLucene

CFLucene is an open source project that attempts to provide developers an easy way to integrate the indexing and searching functions of the Apache Lucene Java library with a ColdFusion web application. The CFLucene is a collection of ColdFusion Components that natively call the Lucene Java classes to index and search any sort of textual data.

http://www.cflucene.org/

The ColdFusion Podcast Episode 18 – Project Management

Simon — Mon, 13 Mar 2006 19:45:35 +0000

Bryan Kaiser & Michael Haynie discuss managing a ColdFusion project in terms of client expectations, project goals and actual development. They also discuss some methodologies to help maximize your development and client relation skills.

Links:

Listen to the podcast or get the feed at feedburner: http://feeds.feedburner.com/coldfusionpodcast