Monday, November 29, 2010

.NET WebClient 403 Forbidden Error

Wasted 2 hours trying to track down the following error when making a simple WebClient DownloadFile request to an Amazon S3 url:
The remote server returned an error: (403) Forbidden.

Offending code and Url:
Uri uu = new Uri("https://zencoder-live.s3.amazonaws.com:443/add9d5d739193c13fcde60d3d7ff5ba7%2Ffe33f4d52e1cff0ef06592ed4041a7dc.mp4?Signature=b%2FXw9ylREb4up4QDw6Tyv9GyQhU%3D&Expires=1291150754&AWSAccessKeyId=AKIAIIEXNN2J4YDTRUVQ");
using (WebClient wClient = new WebClient())
{
    wClient.DownloadFile(uu, @"C:\output.mp4");
}

I was able to use Fiddler to compare Firefox's request versus my .NET application's request.

Firefox:GET /add9d5d739193c13fcde60d3d7ff5ba7%2Ffe33f4d52e1cff0ef06592ed4041a7dc.mp4?Signature=b%2FXw9ylREb4up4QDw6Tyv9GyQhU%3D&Expires=1291150754&AWSAccessKeyId=AKIAIIEXNN2J4YDTRUVQ HTTP/1.1


.NET WebClient:GET /add9d5d739193c13fcde60d3d7ff5ba7/fe33f4d52e1cff0ef06592ed4041a7dc.mp4?Signature%3Db%2FXw9ylREb4up4QDw6Tyv9GyQhU%3D%26Expires%3D1291150754%26AWSAccessKeyId%3DAKIAIIEXNN2J4YDTRUVQ HTTP/1.1

Notice that .NET is escaping my Url. Particularly the forward slash (%2F). You used to be able to pass a dontEscape parameter to the new Uri constructor but now that parameter is deprecated and is always false.

Luckily I came across a workaround on StackOverflow by Rasmus Faber:
Uri uu = new Uri("https://zencoder-live.s3.amazonaws.com:443/add9d5d739193c13fcde60d3d7ff5ba7%2Ffe33f4d52e1cff0ef06592ed4041a7dc.mp4?Signature=b%2FXw9ylREb4up4QDw6Tyv9GyQhU%3D&Expires=1291150754&AWSAccessKeyId=AKIAIIEXNN2J4YDTRUVQ");
ForceCanonicalPathAndQuery(uu);
using (WebClient wClient = new WebClient())
{
    wClient.DownloadFile(uu, @"C:\output.mp4");
}

void ForceCanonicalPathAndQuery(Uri uri){
  string paq = uri.PathAndQuery; // need to access PathAndQuery
  FieldInfo flagsFieldInfo = typeof(Uri).GetField("m_Flags", BindingFlags.Instance | BindingFlags.NonPublic);
  ulong flags = (ulong) flagsFieldInfo.GetValue(uri);
  flags &= ~((ulong) 0x30); // Flags.PathNotCanonical|Flags.QueryNotCanonical
  flagsFieldInfo.SetValue(uri, flags);
}

Now the code downloads the file like without the error!

Saturday, November 27, 2010

.NET Zencoder API Wrapper

I've been working with automated video encoding systems for over 10 years. I'd like to say there have been lots of improvements in the tools available but the only tools that have made life easier are the hosted solutions like Zencoder.

Zencoder has a really nice API builder on their website and tools like John Sheehan's RestSharp make calling web services from .NET easier then ever. But why not take it a step further and get it down to one line of code to submit an encoding job? That's what I've done over my Thanksgiving weekend and the results are on GitHub in my ZencoderWrapper repository.

One liner transcode:
JobResponse job = new ZencoderClient(API_KEY).SubmitJob("http://cdeutsch.com/input.avi", "ftp://ftpuser:[email protected]/videos/", "output.mp4");

You'll need to create an account on Zencoder to use the service. And if you want to encode more then 5 seconds of video you'll need to pick a payment option. You can alternatively use the ZencoderWrapper to create your account:
CreateAccountRequest account = new CreateAccountRequest("[email protected]", "password123");

Here's a more advanced example of creating an Ogg Vorbis file with thumbnails and notifications:
ZencoderClient client = new ZencoderClient(API_KEY);
JobRequest jobRequest = new JobRequest("http://cdeutsch.com/input.avi", new OutputSetting("ftp://ftpuser:[email protected]/videos/", "output.ogg"));
//configure output settings.
jobRequest.outputs[0].audio_codec = AudioCodecSetting.Vorbis;
jobRequest.outputs[0].video_codec = VideoCodecSetting.Theora;
//add a notification.
jobRequest.outputs[0].NotificationSettings = new List<NotificationSetting>();
jobRequest.outputs[0].NotificationSettings.Add(new NotificationSetting(NotificationType.Email, "[email protected]"));
//create thumbnails
jobRequest.outputs[0].thumbnails = new ThumbnailSetting();
jobRequest.outputs[0].thumbnails.base_url = "ftp://ftpuser:[email protected]/thumbs/";
jobRequest.outputs[0].thumbnails.format = ThumbnailFormatSetting.PNG;
jobRequest.outputs[0].thumbnails.interval = 5;
jobRequest.outputs[0].thumbnails.number = 3;
jobRequest.outputs[0].thumbnails.prefix = "thumb_";
jobRequest.outputs[0].thumbnails.size = "120x80";
jobRequest.outputs[0].thumbnails.start_at_first_frame = true;
//submit the job
JobResponse job = client.SubmitJob(jobRequest);

Here's how to check the status of the job we just created:
//get job details.
JobListingResponse job = client.GetJob(job.id);

Here's how to check to the status of the job's output file (there can be multiple outputs):
//get progress of first (in this case only) output file.
JobOutputProgressResponse progress = client.GetJobOutputProgress(job.outputs[0].id);

And finally here's how to get a list of all of your jobs:
//get list of jobs
List<JobListingResponse> jobList = client.ListJobs();

If I can convince Implex (the company I work for) to outsource their encoding to Zencoder I'll be using this wrapper for their QwikCast product. I also have another personal project I plan to use the wrapper in.

If you end up using it shoot me a message I'd love the feedback!

UPDATE 11/28/2010:
Not sure how I overlooked this earlier (either missed the Zencoder "Libraries" link or just got carried away with the idea of trying something new I guess) but Chad Burggraf has already created an excellent C# Zencoder client. I may end up using his library over my own for the asynchronous support. Part of it will depend on if Chad's library can compile with Mono since the personal project I want to build will ultimately be running on OSX. I'm not sure if my library will compile in Mono "as-is", but I believe RestSharp will and I don't think I used anything special beyond what's contained in RestSharp.

Tuesday, November 9, 2010

WatchedIt - An MVC3, Razor, EF4 Code First Production

One of the features I love about applications like Boxee is it treats the content you're watching like email. Instead of "read" and "unread" it's "watched" and "unwatched". This is great if you don't have a significant other who tends to watch all of your shows before you. ;)

So I decided to make an application I call WatchedIt to track where I left off watching shows. What a perfect opportunity to try out some new technologies! For this project I used the following fairly new technologies:

So what did I think of the experience? MVC3 is a promising improvement over MVC2 and Web Forms. Outside of a radical paradigm change (like being more dynamic like Ruby) I'd pick MVC3 for my next project without hesitating. It integrated nicely with jQuery by offering ways to produce and consume JSON.

HTML5 gets a big thumbs up! I've already begun using as many features as I can that are backward compatible with older browsers. If you want to know what you can and can't use now, I recommend HTML5 and CSS3 by Brian Hogan.

I really liked Razor! But the lack of intellisense (auto completion, etc) in Visual Studio was a bit painful. Hopefully this will get added soon and it will be my preferred View Engine hands down. It was a little hard to figure out when you don't have to use the @ symbol to prefix code but the error messages were very helpful.
UPDATE 11/9/2010: And 3 hours after posting this, Intellisense has been added in MVC 3 RC!

EF4 Code First didn't go nearly as smooth as the other new features even though I love the concept. The parts that worked are great but it still has a few too many limitations for me to pick it again without hesitation over other options. Some of the limitations:
  • I couldn't find a simple, elegant way to do a cascading delete of all child objects when you delete a parent.
  • You can't have more then one DbContext share the same database (not a major issue)
  • Deploying to production didn't go very smoothly. I had to configure the connection string to use the "sa" account to get the DB created.
Keep in mind these .NET offerings are all fairly new and I think Microsoft is on the right track with all of them.

I've decided to give back to the community and put the source code on GitHub. I'm sure there are some things I could have done better or that didn't follow best practice so if you find something that would have been much easier to do another way let me know. I'd love to hear it.

Some 3rd party libraries that made this application possible:
Elmah - my favorite error logging library
tvdblib - they did an awesome job of wrapping the TVDB API!
TheTVDB - without this DB this application wouldn't be possible. Please consider contributing info and artwork.

Saturday, October 30, 2010

Custom iPhone 4 Mount For Car

Cutting to the chase. Here are the pics of my new custom iPhone mount!


Previously I had customized my car to mount my iPhone 3G using a Griffin iPhone 3G Windshield Mount.

A couple months later the iPhone4 came out and it does not work with the Griffin mount I was using. I took a gamble and order a Griffin WindowSeat AUX.

I got very lucky and was able to put the cradle from the new Griffin mount onto the existing arm from the 3G mount. It's an extremely tight fit but it works. It doesn't swivel as easily as I'd like but I'm very happy I didn't have to do anymore cutting.

The next customization I wanted to do was to add steering wheel controls for music. I found this Kensington LiquidAUX Bluetooth Car Kit and it was on sale so I got overly excited and bought 2!

After it arrived I was disappointed to find out that my Alpine CDA117 car stereo does not have a regular "line in" that would work with the Kensington Bluetooth kit. :(

In addition the AD2P audio takes over the wired line out so it was looking like I'd have to not use the Bluetooth kit or buy a new stereo. After some googling I found the solution to my problem. If you jailbreak your iPhone and install Bluetooth Profile Selector (BTPS) you can disable the AD2P out and the audio will play out the wired connection, BUT the wireless steering wheel controls still work!

The Kensington Bluetooth kit allows me to play/pause, and skip songs forward and backward and also has a button that will bring up Voice Control. Voice Control seems to work Ok for dialing numbers. So far I haven't found the trick to get it to play songs/artists while connected to the car stereo.

The final mod I did was to disable the "Accessory Connected" splash screen that replaces the iPod apps UI when connected to my head unit. There is another app on Cydia for jailbroken phones called No Accessory Splash. Install this and you can use the iPod app like normal in addition to having the head unit controls work.

The combination of a nice touch screen and steering wheel controls should make my commutes much more pleasant! Especially considering every other feature in my Mitsubishi Evo IX RS is manual (locks, windows, mirrors, seats, etc). For those curious the RS is a stripped down model primarily marketed to be raced. This saves weight and allows the car to be sold at a lower price.

Thursday, October 14, 2010

How Not to Implement Unsubscribe

UPDATE 10/14/2010 11:55 AM CST: SpeakerRate may not have done the unsubscribe correctly. But they know customer service. Thanks guys!
http://twitter.com/#!/speakerrate/status/27356776561

This is just plain unacceptable. I rated one speaker and used my email address to do so. I get a newsletter from SpeakerRate the next day which is fine because I planned to just click the Unsubscribe link at the bottom. Here's what I was presented with:



Do you really think I want to create an account?

Friday, October 8, 2010

Count the Number of Network Connections in Windows

If you need to count the number of network connections to a certain port start by entering this command into a command prompt:
netstat -a -n

Then find the IP:Port combo you want to count and modify the following command to get the count:
netstat -a -n | find /c "<ip:port>"

Example:
netstat -a -n | find /c "127.0.0.1:1935"




Thursday, October 7, 2010

Upgrade ASP.NET MVC 3 Preview to MVC Beta

I previously blogged about upgrading the TekPub ASP.NET 2 Starter Site from MVC2 to MVC3 Preview:
http://blog.cdeutsch.com/2010/08/upgrade-mvc2-to-mvc3.html

Today I upgrade that project to MVC3 Beta.

It was pretty easy. Biggest change was with Dependency Injection. IMvcServiceLocator no longer exits and has been replaced with IDependencyResolver.

My NinjectServiceLocator file has been replaced with NinjectResolver (thanks Jedidja from StackOverflow!) and looks like this:

namespace Mvc3Ninject.Utility
{
    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.Web.Mvc;
    using Ninject;

    [System.Diagnostics.DebuggerStepThrough]
    public class NinjectResolver : IDependencyResolver
    {
        private static IKernel kernel;

        public NinjectResolver()
        {
            kernel = new StandardKernel();
            RegisterServices(kernel);
        }

        public NinjectResolver(IKernel myKernel)
        {
            kernel = myKernel;
            RegisterServices(kernel);
        }

        public static void RegisterServices(IKernel kernel)
        {
            //kernel.Bind<IThingRepository>().To<SqlThingRepository>();
        }

        public object GetService(Type serviceType)
        {
            return kernel.TryGet(serviceType);
        }

        public IEnumerable<object> GetServices(Type serviceType)
        {
            return kernel.GetAll(serviceType);
        }
    }

}

And then in the Global.asax.cs file I changed this line....

//// Tell ASP.NET MVC 3 to use our Ninject DI Container 
MvcServiceLocator.SetCurrent(new NinjectServiceLocator(_container));

...to...

//// Tell ASP.NET MVC 3 to use our Ninject DI Container 
DependencyResolver.SetResolver(new NinjectResolver(_container));

And that's that! Have fun coding!

Here's a complete working sample project based on my Code-First Entity Framework upgrade of the TekPub Starter Site.