Author: Corey

mod_rewrite: URL modification

programming : by Corey — October 2nd 2010, 07:22AM
programmingTo say that there is extensive documentation on Apache and its various plugins, including mod_rewrite, is to grossly understate the term. Documentation is voluminous to the point of beginning to wonder if the various authors had a combined total of more than five dates with actual girls in the history of their lives.

Disgruntled, then, was I to discover that on the entirety of the Internet, there was no documentation surrounding what I needed to accomplish. I've since come to realize that this is likely due to the obscurity of the issue or the availability of other commonly known tools to accomplish the task I had before me.

That task: Rewriting and redirecting a URL from my local environment using Apache and mod_rewrite.

Without delving into specifics, I needed to take an HTTP request generated by a page viewed in my browser and direct that request to another location, including a change of domain.

Now, the more seasoned among you may resolve that the very purpose of mod_alias is to perform this task. However, just because I'm a glutton for punishment, in this particular case, I also need to change the value of a query string parameter in the URL having its domain changed. Gaze upon the domain of mod_rewrite, ye mighty, and despair.

While mod_alias is designed to handle the translation of domains, mod_rewrite is designed to handle that and query string parameters (as well as a bunch of other stuff that I have no idea about). Before we can start directing URLs to and fro, we must first setup Apache.

I'll not regale the reader with the riveting tale of that process as it is rather well (and usefully) documented. The mod_rewrite module must be included in httpd.conf and the Apache instance must be configured to run as 127.0.0.1 on port 80. Do be wary of configuring the server value as localhost because sometimes the value does not translate, especially in Windows. My Apache setup is in Windows 7 x64 with OpenSSL.

The next thing to do is to direct all HTTP requests from the domain that needs to be changed to the local instance of Apache. In Windows, this is easily accomplished via the Windows HOSTS file. An example entry would look like this:

127.0.0.1 ihatecake.neodux.com

This entry will override DNS requests for ihatecake.neodux.com and point it instead to localhost. Apache takes over from there.

With that in mind, let us consider an example URL value:

http://ihatecake.neodux.com/serve?feedme=pie

Clearly, no one hates cake and that pesky query string is returning pie instead of cake, which simply isn't fair. This injustice will not stand; we must modify and redirect this request to attain cake righteousness.

In order to engage mod_rewrite, the traditional method is to add rules to Apache's docroot in the .htaccess file. Because we're not actually dealing with anything in Apache's docroot (despite all the documentation examples using only this scenario), these rules need to go inside httpd.conf. Assuming you've got Apache configured and running, this is quite simple. At the bottom of your configuration file, add a virtual host for your localhost instance of Apache and include the rules to enable mod_rewrite's magic. The first step is to turn the mod_rewrite engine on. It looks like this:
NameVirtualHost 127.0.0.1:80

RewriteEngine on
The next step is to write a conditional statement inside the virtual host instruction set looking for the feedme query string parameter. In Apache, mod_rewrite accomplishes this via regular expression. Now before your panties fully bunch, allow me to assure you that this implementation of regular expressions is very simple once you break it down, so don't let the syntax scare you. The conventions are largely the same as those used in perl, so if you're a perl nerd this will not seem out of place. These condition and rule execution statements accomplish our goal of translating ihatecake to ilikecake and the feedme query string from pie to cake:
NameVirtualHost 127.0.0.1:80

RewriteEngine on

RewriteCond %{QUERY_STRING} ^(.*)feedme=pie(.*)$
RewriteRule ^/.*serve.*$ http://ilikepie.neodux.com/server?%1feedme=cake%2 [R,L]
Let's break it down: The first thing I did is to write a RewriteCond statement that evaluates the query string of the URI for feedme=pie. Most mod_rewrite rules follow the basic convention of RuleName Pattern Pattern, so the RewriteCond rule matches the %{QUERY_STRING} Apache variable against the second pattern. The carat tells mod_rewrite that this is the beginning of a string. The (.*) essentially says, "This means everything before feedme, and since I'm wrapped in parentheses, remember my crap for later." You'll see another usage of that after feedme=pie, so that the entire statement reads [everything before feedme=pie]feedme=pie[everything after feedme=pie]. The dollar sign then signifies the end of the string. In this way, mod_rewrite is able to localize on what you actually want to modify. The purpose, and dare I say the beauty, of the RewriteCond statement is that the RewriteRule statement will not execute unless RewriteCond finds what it is looking for.

The second statement is the RewriteRule. where the URL is modified. Based on what we found (and stored) in RewriteCond, RewriteRule starts its search for the end of the URL, /serve. Again, we see the use of the carat to signify the beginning of a matching string, and the use of .* to mean, "Everything before serve." The dollar sign marks the end of the first pattern string. The second pattern string contains the values we want to change the incoming request to. Since I want to go to ilikepie instead of ihatepie, I can substitute the literal translation here, which I find to be incredibly helpful. After server, where the question mark begins the query string, the usage of the %1 variable says, "Put anything that existed before feedme here." Remember, the condition statement only matches against values found in the query string, so this allows you to rewrite requests that have multiple query strings and/or where feedme is not necessarily the first query string in line. We then substitute the feedme parameter value of pie for cake, and use the %2 variable to include anything that came after feedme.

That's it, right? Close, but there is the matter of the bracketed values out to the right, [R,L]. These are mod_rewrite flags that tell Apache to do certain things. All the flags are covered in the regular mod_rewrite documentation, but R and L are fairly common in their usage. R tells Apache that this should be a redirect, not referencing something in Apache's docroot. If you have a specific need to have the redirect be 301 instead of 302, you can specify that as R 301 or R 302, but it defaults to 302.

The L flag, literally translated, means last rule. It is designed to make mod_rewrite stop looking for additional rules if this rule is successful. This is helpful if you have several RewriteRule statements and don't want to risk having a modified request modified again incorrectly. Also remember that mod_rewrite rules are processed before mod_alias rules and other modules, no matter where they are in relation to each other inside httpd.conf. In essence, the rules are not processed serially.

To test this setup, you can write a local page and view it in your browser. Ensure that you view it via Apache and not from the file via the OS. Using an HTTP request logger like HttpFox for Firefox, you can see the redirect and translation occur in real time.

HTTP request rewriting and redirecting is usually something found in the realm of Squid or other proxy servers. While those tools do work well, novice users (like me) tend to have profound difficulty in getting through the initial setup because those tools are so powerful and have so many options. The setup and operation of Apache, especially as a local web server, is far easier and there are a lot of easy to understand tutorials and other documentation available for it. It's also easier in Apache to lock down access to your instance to only localhost or only your LAN subnet that it is in some other tools.

While this may be elementary to a seasoned Apache guru, I am, by far, not a seasoned Apache guru and it took a great deal of time (and some help from an Apache guru) to make this work for me and to build my understanding of it. Because this specific application of Apache and mod_rewrite was not documented anywhere I could find on the Internet, I though it should be included in the annals of Neodux. Hopefully the next poor fool who needs to do something like this can find it via Google search instead of banging his head against the wall for a week like I did.

Notes for web developers.

rant : by Corey — February 16th 2010, 09:27AM
rantAttention website developers:

Have you spent the last six months living on Twinkies and jet fuel grade coffee to turn out your latest abomination creation? Does it incorporate validated syntax? Does it adhere to all of the W3C standards? Is it fully driven by CSS?

If so, that's great. You've worked hard to achieve what countless Geocities authors could only dream of.

However, this does not give you the right nor the justification to resize my browser without asking first. My personal window settings for my browser are just that: personal. Your content, while possibly revolutionary (but probably wholly unexceptional) is not special, and it does nothing to endear me to whatever your website is trying to tell me.

And woe be unto you if you're trying to sell me something. Not only will I never patronize your site again, I will post rants on websites explaining exactly how you should be publicly flogged for forcing your sizing preferences on the rest of us.

Stop being a butthorn and realize that your, "creatives" aren't anything more than some rendered text and a few pictures.

Kisses,

Mackieman

Multiple profiles in Firefox.

meat : by Corey — November 11th 2009, 04:55PM
meatA large part of my job involves testing websites for various things. Most often, I use a series of Firefox add-ons and some other external tools (proxy server, for example) to perform the tasks I need to do.

Regularly, this involves clearing all cookies and personal data from my Firefox profile. I need to ensure cookies are set correctly and it's tough to do that when all of my regular browsing cookies are there too. Moreover, clearing them all when I need to perform testing requires that I then log into each site I go to regularly again. This makes me a sad panda.

I began browsing the Firefox Add-ons site to try and find an extension or other tool that would allow me to switch profiles on the fly so that I could keep my regular Firefox profile, and all the sites thereunto visited, safely tucked away and ready for my usage at my command all the while maintaining a profile that I could mercilessly abuse in the name of testing.

Of course, such a thing does not exist. However, all is not lost. The Mozillazine has an article on profile management that essentially outlines the method by which the built-in profile manager can be accessed.

As noted above, I am a whiny Firefox user. I do not want to be prompted by the profile manager each time I start Firefox. The solution to this is to have a separate Firefox shortcut with the -profilemanager flag set. I store this in with my regular application shortcuts so that it is easily accessible when I need it. Your profile selection is persistent across sessions, meaning that normal Firefox shortcut will open Firefox with the last profile you selected in the profile manager.

This allows for completely separate add-ons, themes, home pages, connection settings, security settings, and toolbar setups. I've found this incredibly useful in my work and it might service those of you who develop web applications or otherwise tool around on the Internet.

Net Neutrality setbacks.

rant : by Corey — November 5th 2009, 10:24AM
rantFormer presidential candidate Senator John McCain has issued a press release outlining his new bill to prohibit the FCC from issuing binding rules on how the Internet is operated in the United States.

Now, I am a fan of Net Neutrality from a philosophical standpoint. I firmly believe that Internet backbone carriers as well as end user ISPs should not be allowed to implement network practices that limit users based on the amount they pay for service or how much of a service they use.

But, to me, the issue is much broader than basic FCC regulation over the physical network itself. This is a problem for the FTC in how data networks and Internet access is marketed to a strikingly dumb population, and it is an issue for the Justice department to been involved in when companies consort with one another to stifle competition in broadband markets.

Since the early days of 56k V.90 dialup service, ISPs have shouted from the mountain tops that their service offerings were unlimited. Both myself and several of my friends (Bone_Enterprise can attest) took the ISPs to task on this by remaining connected for weeks at a time. Many times, accounts were disabled and we were informed that we were abusing their network. Despondent, we replied that they advertised the service as unlimited. The response was always, "it is unlimited, but you can't use all you want." The incongruity of it all was something that irked us to no end.

As it turns out, what they were advertising is, "unlimited ability to access," and not, "unlimited usage." I believe this is false advertising in sheep's clothing. Regularly the FDA requires pharmaceutical companies to issue new profanely long commercials clarifying their claims about a particular drug and its side effects. Where is this watchdog mentality about the Internet and the trillions of dollars in GDP that it fosters?

The answer is most assuredly money and power. Companies like the toolboxes over at Comcast (which launched this lovely new feature today) throw tens of thousands of dollars at politicians like Senator McCain and his fellow Congressmen (clogged tubes, anyone?) who are uniquely and egregiously unqualified to be making suppositions about global networks, much less binding law.

Were the understanding of how networks function at an organic level and how an Internet free of corporate bullying drives innovation and economic prosperity truly understood by those with the power, I have a feeling that Net Neutrality would've been a reality some time ago. Of course, the blame is not solely on our elected officials; no, the blame is also shared with our fellow citizens. As a nation, and more specifically as a generation, the vast majority of us have become technology gluttons and demand that everything we interact with be simple, perfect, and require little to no effort from us. I believe this attitude is dangerous and stifles the inner need in each of us to grow and better ourselves through learning.

If we can have such stringent oversight of things like pharmaceuticals and stock market actions, things that require significant time and effort to understand and master, why can we not expect the same from the shepherds of our technology? While I generally advocate less government interaction in our daily lives, I do believe that there are some roles in which government must serve, and this is one of them. When companies get so large and their services so vital, they must be cared for to ensure that they do not take advantage of their customers. It is my hope that, in a limited form, Net Neutrality quickly comes to pass and that companies are forced to fairly market their services to an unsuspecting populace.

But something tells me it may be a ways off.

Modifying WLAN adapter status.

windows : by Corey — November 2nd 2009, 08:58PM
windowsToday I started a new job after being out of work for nearly five months. As a part of this job, I've been given a laptop and, like most companies these days, there are both wired and wireless access methods available for the company LAN.

While the access points are secured, the laptops issued to everyone utilize the Windows Wireless Zero Configuration Tool in Windows XP to manage access to the access points in the building. Because I am a task bar minimalist, the ever present icon and information dialogues that appear as a result of connection and disconnection to the access points drive me nuts.

Many current laptops have mechanical switches to enable or disable the WiFi adapter, but the model used by my company is considerably older and predates the widespread deployment of this feature. As such, heretofore, I've always gone into the adapter properties and disabled the WiFi adapter and then followed the same method to enable it the next time I needed it. This process takes two or three clicks and while not difficult, can become cumbersome if you find yourself doing it multiple times during the day.

While taking a break from reading the reams of documentation I've been given, I pondered the question of controlling the adapter hardware via batch script. Lo and behold, Microsoft has created a tool for this very purpose.

The tool is called Devcon and is free to use. It is a command line utility that essentially replaces the GUI Device Manager offered in Windows. There is extensive documentation on it but the important information is thus:

To affect change on a hardware device, you must know its device ID. Devcon provides a function for this called hwids:

c:\>devcon hwids "*" > c:\hardware.txt

This will create a list of all hardware devices currently recognized by Windows along with all pertinent details. In order to not have the information scroll outside the purview of my command session, I dumped it out to a file called hardware.txt.

When the hardware.txt file is opened, you'll see listings like this one for my WiFi adapter:

PCI\VEN_8086&DEV_4224&SUBSYS_10108086&REV_05\4&AD1B67F&0&10F0

   Name: Intel(R) PRO/Wireless 2915ABG Network Connection

  Hardware ID's:

PCI\VEN_8086&DEV_4224&SUBSYS_10108086&REV_05

PCI\VEN_8086&DEV_4224&SUBSYS_10108086
    PCI\VEN_8086&DEV_4224&CC_028000
    PCI\VEN_8086&DEV_4224&CC_0280

Compatible ID's:
    PCI\VEN_8086&DEV_4224&REV_05
    PCI\VEN_8086&DEV_4224
    PCI\VEN_8086&CC_028000
    PCI\VEN_8086&CC_0280
    PCI\VEN_8086
    PCI\CC_028000
    PCI\CC_0280>

The important thing to capture here is the device ID, which will be DEV_XXXX. So in the case of my WiFi adapter, it is DEV_4224.

The next step is to create batch files that have the enable and disable commands in them. You could setup a menu system to select the operation you would like to carry out but I think that defeats the purpose of single click automation. Create a batch file with the following command:

devcon disable *DEV_XXXX*

Obviously, to enable it is the same command utilizing enable instead of disable.

And it's just that simple. Devcon will print some text after the process is complete but launching a batch script with just that command it in will kill the command window as part of the cleanup process. I have both files in my custom quick launch menu and it works well.