A web browser suitable for Harry Potter in WPF!

Daily Prophet eat your heart out! Here is a prototype web browser we threw together in Bling:

We (myself, my intern lighting-export Li SiYu, and a former intern Wang Chao who worked on the browser part) started with code from Chris Cavanagh’s Blog to render a background IE instance into a WPF writeable bitmap, which we optimized using some sample code from David Teitlebaum. The advantage of this approach is that a writeable bitmap is a first class entity in WPF and so can be transformed and distorted, whereas a standard WPF web browser doesn’t support these features due to airspace issues. The disadvantage of our approach is that its a big hack suitable only for playing around (e.g., in the above pic) and we can only update the browser about once every 100 milliseconds, so videos and Flash are extremely choppy (but still very lively!).

The web browser content is projected onto a diffuse material that is lit by a point light. We bind the point light’s position to the mouse so the lighting changes dynamically with user interaction. The height of the light is bound to the slider beneath the address bar. This code sets up the lighting:

ThumbBl LightXY = new ThumbBl();
Action UpdateLightXY = PointBl.Assign(LightXY, canvas.MousePosition);
this.MouseMove += (xx, yy) => UpdateLightXY();
Lighting lighting = new DiffuseMaterialCl().Apply(new PointLightBl() {
  Position = new Point3DBl(LightXY, HeightOfLight),
  Color = Colors.White,
});

This code mostly involves databinding, except for tracking the mouse position, which we do manually. The “Apply” method of material exercises basic lighting equations according to the specified light (diffuse material + point light in our case) to create a lighting result. The lighting code for a diffuse material is encoded in Bling is something like this:

Point3DBl Dir = LightSource.DirectionToPosition(Position);
DoubleBl Dis = Dir.Length;
Dir = Dir.Normalize;
LightingResult = ((LightSource.Color.ScRGB * Dir.Dot(Normal).Max(0) *
  LightSource.UseAttenuation(Dir, Dis))) * Input.ScRGB;

Lights are distinguished by their direction (constant for a directional light, difference between position being shaded and light position for point light) and their attenuation (a bit more math).

We apply the lighting result to the canvas containing the browser via normal mapping, which uses pre-baked texture to add depth and naturalness to the surface being lit. The textures used here make the browser look like it is printed on slightly folded paper! The texture scrolls as the scrollbar is used to scroll browser content, though not at the same magnitude. Code:

Texture HeightField = new ImageBrushBl(HeightImage);
Texture NormalMap = new ImageBrushBl(NormalImage);
HeightField = HeightField.Distort(uv => new PointBl(uv.X, (uv.Y + scrollBar.Value).Frac));
NormalMap =     NormalMap.Distort(uv => new PointBl(uv.X, (uv.Y + scrollBar.Value).Frac));
canvas.Effect.Custom = lighting.Apply((HeightField)HeightField, (NormalMap)NormalMap,
                              canvas.Size.AddZ(0.015d), canvas.CenterSize.AddZ(100d));

Textures in WPF are brushes, so we load our normal mapping files (HeightImage and NormalImage) as image brushes. We distort the resulting textures by shifting them vertically by the scrollbar’s value, which are then converted into proper texture coordinates via Frac (an HLSL intrinsic defined as Math.Abs(Math.Round(n))). We then explicitly coerce the two textures into a height field and normal map respectively, for use in normal mapping. These explicit coercions convert the textures according to a popular normal map and height field texture representations (Crazy Bump and various Photoshop normal mapping plugins). The other two arguments used for normal mapping are the 3D size of the canvas and the eye position we want to use. The code for normal mapping is as follows:

public PixelEffect Apply(HeightField Heights, NormalMap Normals, Point3DBl Size, Point3DBl Eye) {
  return new PixelEffect(input => new Texture(UV => {
    Point3DBl Position = new Point3DBl(UV * Size.XY(), Heights[UV] * Size.Z);
    var eyeDir = (Eye - Position).Normalize;
    var UV0 = UV + eyeDir.XY() * Heights[UV] * Size.Z;
    return this[Position, Normals[UV0], Eye][input, UV0];
  }));
}

A pixel effect is simply a texture-to-texture function that is suitable for use as a pixel shader. A texture is defined as a point-to-color function that can be loaded from a file (as we did with HeightField and NormalMap) but can also be defined programmatically, which is what we are doing here. The math for the resulting texture is beyond the scope of this blog post but is not very difficult: basically we extract heights and normals from the height field and normal map according to the current texture coordinate UV, adjust them for the 3D size of our target, and bias UV by using an eye direction and the current height.  This is then plugged into our lighting result, which in turn is used to compute the resulting pixel from the input texture and the biased texture coordinate.

This effect will work for any WPF application, not just our browser, notice the text field at the top, the slider below that, and the scrollbar tot he side, which are all WPF widgets!

The code for this sample is available in the Bling 3 distribution, which will hopefully be released soon!