Efficient Data Loading with Linq using DataLoadOptions For the last five months I've been working on a hobby project called Mangosteen Nation which is a website for XanGo distributors. I really wanted to use this project to look at some of the new things in ASP.NET and C# such as LINQ and it's been a great experience.

So here is a tip for efficiently grabbing data using LINQ in C# 3.0.

The way LINQ works from what I can tell is that lets say you write a query to get a user object from the database but then you want to access the comments for that user.

MangoData db = new MangoData();
Mangosteen.aspnet_Membership user = db.aspnet_Memberships.First(p=>p.UserId == UserId);
List commentsFromUser = user.Comments.ToList();

This will actually issue two queries to the database although it maintains the same connection. The reason is because it will only return the user object but none of the associated content in other tables. This is good for efficency as we don't need to see the other stuff in the other tables associated to the user but when we need to access that information, it will automatically make another query to the database. If you're not careful, you could end up with a lot of queries to the database that you're not aware of.

What you really want is for just one query to be executed so what we want is that when it returns the user object, it's filled with the comments for the user as well. But there other associations and those shouldn't be returned. So how do we handle this? We use the DataLoadOptions.

If we modify our code with the new System.Data.Linq.DataLoadOptions settings, we'll get the following

MangoData db = new MangoData();

DataLoadOptions dlo = new DataLoadOptions();
dlo.LoadWith(p=>p.Comments);
db.LoadOptions = dlo;

Mangosteen.aspnet_Membership user = db.aspnet_Memberships.First(p=>p.UserId == UserId);
List commentsFromUser = user.Comments.ToList();

The data load options are declaring that whenever an aspnet_Membership object is loaded that the comments should also be loaded as well. Then you can attach that to the database object of your choosing to have those settings applied.

  ¶ 1:27 PM 1 Comments

  Mangosteen Nation Chris Scripca and I just built a new website this last week for Mangosteen Nation which is a site selling XanGo. XanGo is a fruit health drink. We launched the first draft of it tonight. More feature rich versions will be coming in the next month.

Update: This is a closed release.  ¶ 4:47 AM 0 Comments

  Summary of my BarCamp expierence
I had a great weekend up in Palo Alto and San Fransisco this weekend. Got to meet a lot of really nice and smart people at BarCamp which was a great experience. You can't find a nicer group of people all trying to learn and help each other out.

Saturday I flew into San Jose and went straight BarCamp in Palo Alto. After wandering the streets of Palo Alto for a bit trying to get my bearings, I finally found another attendee, Adam Gottesfeld from Palomar Ventures. The green name badges everyone wore really helped to identify who was a part of BarCamp.

After all of the sessions for the day there was a party at the Blue Chalk Cafe where everyone was getting free drinks. They were also doing lightning round presentations of web applications on a big screen that people there were working on. It's always interesting to see the kinds of things the developers there are building.




I ended up staying the night at the Social Texts offices working on some development stuff and trying out Selenium, a web UI testing application. A couple of other guys were there as well working on stuff. One guy, Action Merchant, was building the Zoo Facebook application.

Sunday the sessions were longer and I attended two of the Facebook application sessions which were good. The last couple posts contain the notes from those sessions. I think those two sessions had the most people of any of the sessions that I attended of both days. There is a lot of interest in what people can do with Facebook apps and still remains a lot of confusion about how they'll actually fit into the business environment.  ¶ 4:50 PM 2 Comments

  Facebook application creations at BarCamp session
These are my quick notes of everything we talked about during the presentation.

For the most part it seems to be a shotgun style approach to the application development. Much of the discussion is with the creator of the Facebook app Zoo.

There was a question about the business sense of Facebook apps but what if this is like the web circa 1995. It's good to be there to shape what it becomes in the future and there will be business utility but we don't understand what it is yet but it will be come apparent in the future. When it does become apparent to everyone, the Facebook app creators will be well positioned.

One of the audience has been using Facebook like a Sharepoint site so that the 17 people at his company can communicate and share documents.

Graffiti is interesting because it's become a popular Facebook app but is similar to a child's game. It's a basic concept that people enjoy. I think part of it is that people want to modify or add their own unique mark on another person's profile.

One of the people here has a company that is working on about 20 applications, some of them are throw away apps but then some of them are more about productivity applications so that people can collaborate together. He hasn't launched any yet.

One interesting note, apparently Facebook time is 100X internet time. I don't know what that means though.

For a lot of the Facebook development, we want to figure out if the massive amount of early development that we saw at the beginning when the application development tools came out is going to stick. A lot of the people doing it are just trying to learn something and they aren't so concerned at this point about making money. The money will come at some point. You know, step 1: steal socks, step 3: profit. I believe traffic will be moved to other sites through the applications instead of the applications being businesses themselves.

There was one question about possible collaboration between working together on applications or maybe making the applications collaborate together. There is more value in using a set of apps.

There is an IRC channel where a lot of the Facebook users hang out that would be a good way to promote or get help on development.

irc.freenode.nut #facebook

There is also a meetup group in Silicon Valley that also does hack-a-thons every few weeks. There is a Facebook group for the Facebook groups.

Some of the Facebook applications have been open sourced but it's not too big because initially Facebook application developers were getting bought out after only a week of work. So many of the developers thought that maybe they should try to get bought out as well.

How are people finding applications? It seems some people are finding them through news feeds but also some people want to do something specific so they search for the applications. I've found all my apps by my friend's news feeds.

What are the downsides of Facebook applications? The types of information appearing in a news feed is becoming somewhat spammy. Things like someone biting another person as a zombie with an app isn't really going to help others keep track of what is going on in those people's real lives. The news feed only contains so much and a bunch of Facebook only actions are going to clutter the feed.

There was a big discussion lately online on this topic that iLike and many of the popular applications use the spammy tactics for posting to the Facebook news feed. Facebook will kick the application if it spams the news feed without user interaction but even with user interaction, it's still can be annoying to see.

The Facebook photos application was really successful because you could tag your friends in photos. Even though only 1/3rd of people in the Facebook community had added photos, most of the Facebook community had been tagged in a photo.

How do you choose which language to use? PHP is what Facebook supports first and is also what Facebook was written in. Everything should work best in PHP.
What is the ecommerce model for applications? Some of the apps are having people fill out surveys and as a reward they give the user digital rewards on Facebook.

Another thing is affiliate links from Amazon and another that works with iTunes.

How do you design a succesful API and why is it so awesome for Facebook? Facebook hasn't given out any contact information through their API. So it hasn't given out any email addresses or screen names which is good for privacy. But they're giving out all of the other information such as the networks the user is a part of.

What categories of apps are people doing in the room? Comparing photos, business productivity, dating, web presence, a sorting tool by looking at a friends list for scheduling events, professional networking.

Are people working on business productivity and professional networking applications?

Why are we developing for Facebook? Facebook is one of the only sites large enough with an API good enough so that we can actually develop awesome customized features for.

Many of the questions at the forums for developers at Facebook have better answers often.

Best suggestion? Hang out with people that do Facebook apps. Get in the wiki and the IRC.

How many people actually use Facebook to track their friends in this room? Maybe 25% of people raised their hands.

Business productivity apps? The definition of friend on Facebook is very loose so it's hard to categorize friends as coworkers as well. As a result, there are some applications potentials there but people aren't going to Facebook yet to do work.

Companies like Google with Google Docs are probably much better positioned to build the productivity apps than a 1 man team. One guy spent weeks building a complex documents app and it didn't take off. So now he's building a simple photo rating app or something similar.  ¶ 11:11 AM 1 Comments

  Facebook presentation 1 at BarCamp I was here at BarCampBlock all last night with a couple other guys. I was working on some ShareFlo stuff, one guy was working on the Zoo Facebook application that he wrote and I'm not sure what it was the other guy was working on. But it's been awesome to be surrounded by this much creativity. The people at this event are incredibly passionate about creating startups and building cool things.


I'm currently in a Facebook applications session with a recent Harvard graduate Jeff Gibb discussing some Facebook applications he's built. He built a simple little app last week where a person could pick a picture of an dinosaur and he sent the app to his brother. Since then, more than 500 people are now using the appliation.

But someone in the discussion just raised a good point that we need to know who is accessing our Facebook app. So what he has done is embed a Google Anlytics HTML so that he can get a sense of where the users are coming from. But getting engagement metrics on a Facebook app is difficult and is something that might need to be built by the application builder themselves.

One problem is the TOS for a Facebook app says that you can't run advertisements in your Facebook app. So how do you make money off of it? I think the only way it is beneficial to a company is if the Facebook app isn't pushing ads but is an advertisement itself. Sites like HotOrNot and other companies are using Facebook apps as traffic generating tools.

There is a great opportunity to use Facebook to give away almost a sample of your site.

The first version of the Facebook API was designed for a user to access their data from a third party site. There is always the question for a Facebook app developer is trying to figure out how much to give away.

Chris Messina is talking about how he enjoys using Twitter because it interacts with the outside world and when he was at Magnolia and allowed people to create a Magnolia account within the Facebook app. When the account was created, the data was being pumped out of Facebook. When they finally "grow up and move into the real world" they can go onto a new site or maybe move onto Magnolia.

Facebook has kind of like their own Javascript because of security issues. The reason you're limited in what you can use in Javascript is so that you don't mess with the rest of the page. Facebooks HTML formatting is actually called FTML. Even just creating a clock took a long time because you can't autoplay a flash easily.

Someone just brought up an interesting point that the security problems in Flash are bigger than the ones in Javascript but I can see why Javascript was locked down so much. MySpace was once taken down for a day or two because of a person that created a Javascript hack so the additional security is warranted. Later I'd like to see the sandbox environment improved.

The privacy discussion just came up that installing all these Facebook applications can be a major security issue. No one is sure if Facebook is thinking of privacy of a users data much when it comes to an application. Keeping all those check boxes checked can be a privacy issue for people.

If you're interested in more information about Facebook development, go to Meetup.com for the Facebook developers group.  ¶ 10:23 AM 0 Comments

  BarCamp Evening It's late at night at BarCampBlock and the party is over. The party was after all the sessions at the Blue Chalk Cafe in down town Palo Alto. I got up this morning at 3AM and it's been none stop but a lot of fun today

I attended some really good discussions today and learned about a couple of new tools that I think I'll be using in the next few days for sure. One was what the last post was about. The other tool was GreaseMonkey which was a tool for manipulating the display and function of an existing website.

Tomorrow there are even more sessions. I'm not sure on what yet. Some of the people are staying over night in the SocialText offices. It's a pretty big office with not much in it so I'm wondering what they use the room for normally.

Oh, cool, Robert Scoble is here...  ¶ 10:19 PM 1 Comments

  BarCamp: Session on Selenium by Nelz It's lunch now today at BarCamp. I've met a bunch of great people and been in two really awesome sessions so far. The second session I went to was Nelz's session on Selenium which is a tool for Firefox for testing a website. As I was sitting through the presentation I took down some notes and I'll just drop them below. Forgive me for the errors.


Selenium <-- AKA: a cure to mercury poisoning.

1. Runs across OS's and browser.
2. Uses Javascript and iFrames to embed a test into your browser. Drives your app. Interact with JS to emulate user actions.
3. It can control starting the browser and closing it. AUTOMATED TESTING!
4. Selenium IDE
1. Has debugging
2. Easy selection of on page elements
3. Recording and playback of the tests. Just start the recording, step through the pages and then save the recording. The script of the steps can then be passed onto other developers easily or other people with Selenium to do the test themselves.
   
4. Saves tests off in multiple formats such as Ruby, PHP...
5. Shortcomings of IDE
1. Firefox only
1. The recorded session can be played in any browser.
   
2. Sometimes thrown off by AJAX events.
6. No one does RAILs at this presentation.
7. Tests can be handed off to other people so that they can run the test themselves to see the issue as well. Great for showing the problem.
1. A script of the test can be exported as Java, C#, HTML, Python, Ruby so that Selenium Remote Control can run the script as part of the main tests.
8. Doesn't to browser version testing.
9. "Click and Wait" can have a wait time set for it.
   
10. J-Meter can be used for performance testing.
11. Fitness is another testing tool.
    
12. Look into Floyd for testing browser tests. It speaks to the browser via the browser API.
1. Javascript has limitations because of the security.
   

  ¶ 1:30 PM 0 Comments

  PostgreSQL SQL function definition tables Last night I was creating a code generator using Code Smith that would create C# functions for every database function in a schema but I needed a SQL statement that would get me the definition of the function. It wasn't obvious how to retrieve the parameters and the return type for the already existing functions that were in the database. At first I thought I would be able to find a column in the database that had the DDL and parse it out for the values I needed but no such column existed.

So here is what I discovered. When a database function is being added, PostgreSQL parses out the database function parameters, return type, namespace, name of the function and body and inserts them into some tables and columns. When the parse of the parameters is done, the parameter names are placed in one column surrounded on the ends by brackets and the parameter types are placed in another column. When the parameter types are stored, they are stored as their numerical representations instead of the names like "integer" and "numeric". Below is a list of some of the fairly commonly used types and their numbers.

• 1043 char varying
• 21 smallint
• 1114 timestamp
• 23 integer
• 1700 numeric
• 20 bigint
• 25 text
• 1082 date
• 16 bool
• 869 inet
• 2279 trigger
• 1790 set of refcursor

If you want to look at a list of all of the functions for a schema, the following SQL query will return the data needed. This SQL query was mainly derived from looking at EMS.

SELECT p.proname AS name, p.oid, p.proargtypes AS args, ds.description , p.prorettype AS rettype,
p.proretset, p.probin, p.proisstrict AS strict, p.prosrc AS body, l.lanname AS lang,
u.usename, p.prosecdef, p.provolatile, p.proisagg, n.nspname, proargnames, p.proargmodes, p.proallargtypes
FROM pg_proc p
LEFT OUTER JOIN pg_description ds ON ds.objoid = p.oid
INNER JOIN pg_namespace n ON p.pronamespace = n.oid
INNER JOIN pg_language l ON l.oid = p.prolang
LEFT OUTER JOIN pg_user u ON u.usesysid = p.proowner
WHERE n.nspname = 'public'
ORDER BY p.proname, n.nspname

"nspname" is the schema name that the functions will be pulled from. The parameters list that has the integer type definitions will be in the "args" column and the parameter names will be in the "proargnames" column. The "body" column contains the body of the function.

I will try to post the Code Smith template that I created for generating the C# classes soon.  ¶ 5:39 PM 1 Comments

  Growth of Twitter Response
Kottke wrote an interesting post today analyzing the growth of Twitter versus Blogger and had some graphs that I thought were interesting.

From the graphs, Blogger has had an amazingly stable growth rate since it's time of inception. I'm really amazed how how consistent their growth has been in the number of messages that are posted with it. (I use blogger as well for my blog.)

For Twitter, it's growth in messages sent was massive over the last few months. Although it's far easier to send a message via Twitter because they're smaller than blog posts, it looks like a lot of people that are signing up for the Twitter service are sticking with it right now. I'm fairly skeptical when it comes to the success of mobile applications like Twitter so I'm surprised at it's level of growth it has been able to achieve. I also wonder what their revenue model is.  ¶ 6:42 PM 0 Comments