NullifyNetwork

The blog and home page of Simon Soanes
Skip to content
[ Log On ]

Archive - Historical Articles

You are viewing records from 05/16/2009 21:32:20 to 03/03/2010 09:26:03. I'll be adding support for selecting a date range in future.

I was experiencing problems with SQL compact edition (complete .NET framework freezes) and found that not only is it not thread safe (fairly obvious) but it's doesn't like to even have multiple instances of itself on the same thread - it deadlocks a lot if it does.  I also found that if it runs out of instances then it unloads from memory and has to be reloaded again afterwards, so avoiding that can result in a many tens of thousands of percent performance improvement.

So here's a small class I wrote to track the connection per thread based on someone elses on the Internet (they were just creating one per thread, in this one it is creating one per thread and maintaining the lifespan of that connection to close it when the thread it was for has been closed).  I'll probably add per-connection-string pooling but figured that the important thing is the concept of the solution to the COM deadlocks and total freezing of the framework in this.

/// <summary>
/// Manages open connections on a per-thread basis
/// </summary>
public abstract class SqlCeConnectionPool
{
    private static Dictionary<int, IDbConnection> threadConnectionMap = new Dictionary<int, IDbConnection>();

    private static Dictionary<int, Thread> threadMap = new Dictionary<int, Thread>();

    /// <summary>
    /// The connection map
    /// </summary>
    public static Dictionary<int, IDbConnection> ThreadConnectionMap
    {
        get { return SqlCeConnectionPool.threadConnectionMap; }
    }

    /// <summary>
    /// Gets the connection string.
    /// </summary>
    /// <value>The connection string.</value>
    public static string ConnectionString
    {
        get;
        set;
    }

    /// <summary>
    /// Gets a connection for this thread, maintains one open one of each.
    /// </summary>
    /// <remarks>Don't do this with anything but SQL compact edition or you'll run out of connections - compact edition is not
    /// connection pooling friendly and unloads itself too often otherwise so that is why this class exists</remarks> 
        /// <returns>An open connection</returns>
    public static IDbConnection Connection
    {
        get
        {
            lock (threadConnectionMap)
            {
                //do some quick maintenance on existing connections (closing those that have no thread)
                List<int> removeItems = new List<int>();
                foreach (var kvp in threadConnectionMap)
                {
                    if (threadMap.ContainsKey(kvp.Key))
                    {
                        if (!threadMap[kvp.Key].IsAlive)
                        {
                            //close the connection
                            kvp.Value.Close();
                            removeItems.Add(kvp.Key);
                        }
                    }
                    else
                    {
                        kvp.Value.Close();
                        removeItems.Add(kvp.Key);
                    }
                }
                foreach (int i in removeItems)
                {
                    threadMap.Remove(i);
                    threadConnectionMap.Remove(i);
                }

                //now issue the appropriate connection for our current thread
                int threadId = Thread.CurrentThread.ManagedThreadId;

                IDbConnection connection = null;
                if (threadConnectionMap.ContainsKey(threadId))
                {
                    connection = threadConnectionMap[threadId];
                }
                else
                {
                    connection = new SqlCeConnection(ConnectionString);
                    connection.Open();
                    threadConnectionMap.Add(threadId, connection);
                    threadMap.Add(threadId, Thread.CurrentThread);
                }

                return connection;
            }
        }
    }
}
Permalink 

My integration/automation system 'Zorganiser' is almost ready for a closed beta test (albeit with a limited set of features but enough to be useful!).  It'll let you control automated tasks on any number of computers, so you can do things like tell 50 webservers to install a particular application, run a particular Windows Workflow or to just shift data around between your installed agents - for example to take data from one business application and deliver it to another in a slightly tweaked format regardless of where the two applications are.  You can also configure one event to trigger another - so the workflow could poll something and then trigger an event across all your workstations or servers.

I already have a few people who are interested in testing it, but if you're also interested please e-mail me at (redacted)!

Permalink 

Due to a variety of my circumstances changing and the way the exchange rate was fluctuating I changed from having all-US servers to using rented dedicated servers from 1and1 internet - billed in UK pounds. Right now it's particularly appropriate with both the Euro and the Pound hugely devalued against the US dollar.

I didn't expect much - the plan I picked was half literally half the price I was paying before but I got a higher specification server, unlimited bandwidth with no concern about overages anymore or anything of the sort and a significant improvement on the latency to my Texas based servers (which is understandable given the speed of light, but the network is also less congested and I can regularly get 85Mbps and less than 20ms to the UK when testing it).

So I've been with them almost a year now and the availability has far exceeded that of my previous datacenter - I've had about 2 minutes downtime the entire time, so I thought I would mention it as it's relevant to a project I'm working on at the moment...

Fortunately that does mean I've not managed to see if their support is any good, but the service certainly is.

Be warned that their network setup is not really conducive to VMWare or virtualisation in general - they use 1 ip sized subnets and gateways that you can always access even though they aren't in your subnet through some TCP/IP stack tweaks (I think it's "All subnets are local" combined with the switches having some custom configuration, but I've not yet figured it out and it's the first time I've seen something similar done).  Their network does have a nice Cisco switch based firewall for every custom because of the strange setup though, and it lets you have public IP's from different subnets so it's got its advantages (no renumbering!).

Permalink 

Just a very quick reminder to myself that the file dialogs filter string (OpenFileDialog.Filter and SaveFileDialog.Filter) format it is "Description|*.ext;AnotherDescription|*.txt" etc.

I always seem to get this around the wrong way for some reason.

Permalink 

I'm trying to use a Linq datacontext from a partial trust environment (well, after Assert'ing my way back up) and I'm getting the spectacularly verbose error:-

System.Runtime.CompilerServices.StrongBox`1..ctor(System.__Canon)

Which looks like it's lacking an actual human-readable error but at least it's distinct!

In my case rather than asserting the individually needed parts I had to assert full-trust from my intermediary class (the one that is signed by a strongname to allow it to always be full-trust in my custom AppDomain - see my article on creating a restricted AppDomain - see one of the comments in the code example) between the untrusted plugins (things inside the appdomain with the restrictions applied can call into this and anything the intermediary class does is unrestricted but even things it calls get the inherited permissions from the caller) and the Linq data context user:-

PermissionSet set = new PermissionSet(PermissionState.Unrestricted); //fulltrust especially for Linq :(
//removed my carefully crafted assert set for Linq/SQLCE in exchange for the above being unrestricted:-
//set.AddPermission(new FileIOPermission(PermissionState.Unrestricted));
//set.AddPermission(new AspNetHostingPermission(AspNetHostingPermissionLevel.Unrestricted));
set.Assert();

//Make the call to whatever uses the linq datacontext

Which was a little annoying, but worth knowing.  (The assertion will revert when the stack pops from the method this is in, however do be aware not to allow anything under this method to allow the plugin to execute a delegate or something).
 
I did find a possible workaround of moving a member variable I was accessing into a local variable in the Linq query, but this didn't work in my case where asserting this did.
 
Permalink 

I'm sure there's an easier way to do this, but I've been maintaining my own class to do the job for a while now and keep needing to find it in my code library every time I use it - I figure it might be useful to others:- 

using System;
using System.Collections.Generic;
using System.Text;
using System.Data;

namespace CSVParser
{
    /// <summary>
    /// Simon's stock CSV parser class
    /// </summary>
    public class CSVParser
    {
        /// <summary>
        /// Takes the CSV files contents and creates a data table from it
        /// </summary>
        /// <param name="csvFileContents">The entire contents of a CSV file split by line (rather than the filename)</param>
        /// <param name="validateLayout">Validate the file to check conformance of the layout with expected standards</param>
        /// <param name="topRowIsHeader">The top row is the header row, take it as the column names rather than data</param>
        /// <returns>A datatable</returns>
        public static DataTable DataTableFromCSV(string[] csvFileContents, bool validateLayout, bool topRowIsHeader)
        {
            DataTable outputDataTable = new DataTable();
            List<string[]> csvFileRows = new List<string[]>();
            List<string> columns = new List<string>();

            #region Pre-parse the file
            int columnCount = 0;

            bool gotHeaders = false;
            int rowNumber = 0;

            foreach (string line in csvFileContents)
            {
                string[] parts = ExtractCSVElements(line);

                //initial set of header names but only if the top row is header option is set
                if (!gotHeaders && topRowIsHeader)
                {
                    columns.AddRange(parts);
                    columnCount = parts.Length;
                    gotHeaders = true;
                }
                else
                {
                    if (parts.Length > 0)
                    {
                        csvFileRows.Add(parts);
                    }
                }

                if (parts.Length > columnCount)
                {
                    //if set to validate the layout and that the first row contains the headers then we know any extra columns are wrong
                    if (validateLayout && gotHeaders)
                    {
                        throw new Exception("Row has extra data columns: " + rowNumber.ToString());
                    }

                    //new column detected mid-data-set!
                    for (int i = columnCount; i < parts.Length; i++)
                    {
                        columns.Add("Column " + i.ToString());
                    }

                    columnCount = parts.Length;
                }

                //we always ignore zero length rows as the last line can be empty
                if (parts.Length < columnCount && parts.Length != 0)
                {
                    if (validateLayout)
                    {
                        throw new Exception("Row has missing data columns: " + rowNumber.ToString());
                    }
                }


                rowNumber++;
            }

            #endregion

            #region Build the data tables layout and data

            //columns
            foreach (string column in columns)
            {
                outputDataTable.Columns.Add(column);
            }

            //rows
            foreach (string[] row in csvFileRows)
            {
                outputDataTable.Rows.Add(row);
            }
            #endregion

            return outputDataTable;
        }

        /// <summary>
        /// Extract the elements of a line from a CSV file with support for quotes
        /// </summary>
        /// <param name="line">The data to parse</param>
        private static string[] ExtractCSVElements(string line)
        {
            List<string> elements = new List<string>();

            //do the initial split, based on commas
            string[] firstParts = line.Split(',');

            //reparse it
            StringBuilder temporaryPart = new StringBuilder("");
            bool inside = false;
            foreach (string part in firstParts)
            {
                //are we inside a quoted part, or did we just find a quote?
                if (!inside //we're not inside
                && (!part.Contains("\"") //and we don't contain a quote
                 || ( //or we're handling a single quote enclosed element
                        part.StartsWith("\"") //we start with a quote
                        && part.EndsWith("\"") //and we end with a quote)
                    )
                )
                {
                    elements.Add(part);
                }
                else
                {
                    if (inside)
                    {
                        //we are still inside a quote...
                        temporaryPart.Append("," + part);

                        if (part.Contains("\""))
                        {
                            //then we are also at the end!
                            elements.Add(temporaryPart.Replace("\"", "").ToString()); //add the part minus its quotes to the array
                            //all done!
                            inside = false;
                        }
                    }
                    else
                    {
                        //else we just found a quote!
                        inside = true;
                        temporaryPart = new StringBuilder(part);
                    }
                }
            }

            return elements.ToArray();
        }
    }
}

 

Permalink 

Edit: Due to some investment problems I can no longer employ anyone, sorry :(

My new company - Zorg Solutions Ltd is now looking for at least one software developer to fill out a team, preferably you'll be an enthusiastic multi-skilled person but we're flexible and will distribute work based on what you can do.

Do you have a background as a C# software developer that has done either Winforms or ASP.NET in the past?  You will need to have at least a couple of years of experience developing with the .NET framework but I will be willing to take hobby or open source work as experience if it's provable that you did it.  This could be a great opportunity to jump into the software development world for someone - there's no requirement for a degree or other certification.  Any experience of Windows Azure, Silverlight or WPF would be an advantage.

Or do you have any experience (hobby or professionally) in electronics, firmware, C/C++?  We have contracts for some hardware development and the opportunity to work with and program the hardware we'll build - probably Atmel microprocessors or PIC microcontrollers, but even some Linux on ARM9 based chips.

If you answered yes to either of these questions and would be willing to commute to Basingstoke then I am interested in hearing from you!  Any experience of the following would make you stand out but is not essential:-

  • SQL experience (preferably SQL server's T-SQL dialect)
  • Windows Workflow Foundation
  • Biztalk
  • Silverlight
  • Mono

Salary will be negotiable (but as an idea we would be willing to pay either side of £35k) and based on experience.  Working hours and benefits will be fully negotiable - we can be very flexible and understand if you have personal commitments.  Training and an MSDN subscription will also be provided!

If you apply (just send me your CV) or contact me now to discuss in more detail then you will be ahead of the conventional advertisements for these roles - we'll be putting them out conditional on responses to this posting and a few other enquiries - so you have nothing to lose!  No recruiters yet please!

Edit: Due to some investment problems I can no longer employ anyone, sorry :(

Permalink 

I finally set up my new companies website (thanks to the snow here totally blocking the ability to get to the office!) - it's lacking content at the moment but the design is in place and I think it looks excellent and embodies the company quite well.

http://www.zorg.co.uk/ 

What do you think?  Anything I should change?

I went through the images on fotolia for quite a while before settling on that one for the front page.  We're using SIP phones for the telephony and I am hoping to have my automation system at the alpha stage within a few months so I'll be looking for people to test it before we go public with a beta (Google style!).  I already have a prototype Workflow designer but at the moment there's not enough to use in production yet.

I am hoping that this product will help in a lot of situations, everything from automated deployment of complex systems, testing your software without needing to sit there and do it manually to automating business processes (or virtual machine operations like creation, start-up and shut down!).

All as a simple and easy on-line service you can optionally subscribe to (there will be a free offering to get to try it!).

Permalink 

Today is my last day at my current employer, a company called EasyTrace which has gone from a few people in a cow-shed just outside Basingstoke selling a DOS product to a market leader selling modern software I can feel proud to have written the vast majority of.  Over a year ago now they were also bought by RM PLC and the RM people I have worked with have been enthusiastic and fun to interact with.

I have been there for over five and a half years and would like to wish all the present and ex-employees, suppliers and RM people who have helped out over the years the very best for the future.

Everyone has been amazing and put in a lot of long hours and effort to make a happy client base of schools, colleges, hospitals and big companies.  It has been a wild ride, and it has been a pleasure working with everyone.

I will be providing some help to the company after the relocation to the new offices in Abingdon this Christmas but I am mostly taking this opportunity to set up a new business and have some more relaxing fun for a while exploring an idea I have.  It also hopefully means I'll have the opportunity to get to some conferences and write more technical blog posts - I'm fully aware I haven't made any decent ones since May as I have been spending a lot of time on work projects even out of hours recently.

So, here's to the future!

Permalink 

This is an excellent usability study comparing the major webmail providers sign-up forms out there, it almost certainly has an applicability to all software development to a degree:-

http://www.cxpartners.co.uk/thoughts/web_forms_design_guidelines_an_eyetracking_study.htm 

So take a look.

It covers things like making name fields appear as one entry so they require less thought to enter, and making mobile phone fields not take an age to decide what bit goes in which box.  There were also some good basic layout hints and talk of when to use groupings and when not to!

Permalink 

With development getting more and more complex and thousands of lines of code being required to do things that used to take a couple of hundred, it's nice to see a simpler approach to things.  Sometimes the complexity has great benefit (like reuse and flexibility), but when learning that isn't necessarily first on your mind...

I just noticed a blog post on the Microsoft Small Basic blog and have since had a quick try... It looks like an awesome educational tool - easy to use and I certainly remember playing with logo and basic when I was younger - this provides both of those combined (along with decent graphics and no overhead).

So the blog is here:-

http://blogs.msdn.com/smallbasic/

And the program itself (which is basically a tiny IDE) is at http://smallbasic.com/ - hopefully it will let a new generation of children find programming as fun as it was for me when I was young.

Edit: Yes this is no match for C# which is a wonderful language and has a vast framework beside it, but this IS able to make a logo turtle move around in three lines on screen...

Permalink  3 Comments 

For those that don't know me, I work in an education sector linked business - which means that when the schools/colleges and universities go back we always get a vast increase in the amount of calls and number of issues that people raise.

So I've been trying to make up for the development time that was lost doing support by working later here and there.  Thankfully September is now over so I can return to having a personal life and possibly doing more research/blogging!

I have several projects I am investigating right now and hopefully one of them is getting closer to being something I can talk about and that will be able to have a bit of an ecosystem sit on top of it.  For all those people that were disappointed that Oslo wasn't a platform, this might make up for it, depending on what you were expecting :)

Permalink 

I am looking to merge both a P2P WCF service (for LAN support and accessible hops) and a publish-subscribe server to coordinate everything, this is just a post to keep track of the URL's related to doing this as I find them :)

WCF can't use its PeerChannel to connect to both hosts on the internet and local network at the same time, so there'll need to be two PeerChannel's and a Publish-Subscriber service bus connection to work out where everything else is.

Publish-Subscriber: http://msdn.microsoft.com/en-us/library/ms752254.aspx
PeerResolvers: http://msdn.microsoft.com/en-us/library/aa702771.aspx
Writing a custom peer resolver (new one needs to handle non-local hosts by requesting from the publish subscribe service): http://msdn.microsoft.com/en-us/library/ms751466.aspx
Sample WCF P2P chat app: http://msdn.microsoft.com/en-us/library/ms751502.aspx

(This post will be updated with more links as I find them)

Update: What I didn't realise when I started looking into this is that the PeerResolver service is not a magic 'find everything on your network' concept, but rather just a WCF service you register and unregister at, so it's basically already the public-subscriber list I was looking at.  That leaves the question of if the PeerResolver service does that, how on earth is it supposed to find things on the local network?  Well that would be the PNRP service, which happens to not be compatible with loads of OS' and isn't installed by default.  Great one there guys!

So instead I'll just make my own UDP transport for the peer resolver, and if anything I'll get to learn how to make a new transport :).

Permalink 

So this is a little different to my normal posts that are all 100% technical, but via the Omnima site (company who sell some embedded components) there's a link to a review of one of the products they sell which is used for monitoring stuff, which links to a specialist site on aquariums.

Well, it has some amazing photos of reefs on it which I thought were worth linking to.

Permalink 

Downloading Windows 7 slowly from MSDN since around 6.30PM here and it has been getting gradually slower.

So please wait before downloading it ;)

Edit: And Server 2008 R2 is out too as of the 14th August 2009!

Permalink 

This is a moan mostly, but has anyone else noticed that the developer story for silverlight development is totally utterly rubbish?

I had hoped that Microsoft would fix it for Silverlight 3 so that developers had a first class UI editor rather than having to go use a trial copy of Expression Blend with the worst user interface I have ever come across (I have MSDN but there's only ever an old version of it available and the current version appears to always be the CTP/Beta), or some third party app just to put down some buttons (with the 'run an app locally' addition and loads of other great features why has no time been spent on a decent design-time experience?).

I don't want to write XAML, I wouldn't mind adding controls using some C# code but I really hate XML, it's neither intuitive nor functional (can't easily call/make functions to draw X number of controls but must declare them individually).  I might be able to do Silverlight using 100% C# but it still comes down to the point of why is there no decent designer!???

I know this is supposed to be fixed in Visual Studio 2010 but it's really annoying.

Update: Now it's released I've had a chance to play with Expression Blend 3 and it's quite a bit better than the previous incarnations, and controls are now visible in a seperate section, so I think they are let off there as they've fixed the UI problems quite a bit; but the developer story for Silverlight is still terrible.

Permalink 

My ASP.NET server/composite controls were not getting events from child controls, but only on the real pages, they worked fine on test ones.  It turned out that I had a master page enabled but didn't have an ID set for it, so it was auto generating it.

Just add this to your master pages code file:-

protected override void OnInit(EventArgs e) 
{ 
	base.OnInit(e); 
	this.ID = "SomeName"; 
}

And child controls will start working as expected.  I don't often do web development so this stumped me for quite a while...

Permalink 

This is a story of how to handle a customer who could have ended up sending something back and getting expensive.

So: I've been interested in getting an Aeron chair or Freedom Task Chair or similar for a while now and only been put off by the price.  Eventually Amazon being the amazing pool of data it is made a suggestion of a similar one called a Cobham mesh back chair, which looked like it would fit the bill but only cost £270.

I obviously googled for more information, and found that whilst there weren't any reviews there was a place called chairoffice.co.uk selling the exact same Cobham mesh back chair for almost a hundred pounds less than the other suppliers out there (around £150 with VAT and delivery, or £119 ex VAT).  A quick background check to see if they were legitimate showed they were based really close to the actual manufacturer - Teknikoffice and happened to have their site hosted on the same server which explains the price difference a bit!!  I ordered after a particularly bad day at work in a sort of shopping to make myself feel good mood.

I was really impressed when it arrived the next day after ordering (very speedy dispatch), and I got right down to assembly after work - which was remarkably simple compared to a few other chairs.  Right after assembling it though I sat down in it and tried to adjust the headrest - and noticed that the back part had a hairline fracture, which when the headrest was extended was completely sheered through in shipping!!!!  I was gutted as the chair was really comfortable and I had that sick feeling of looking forward to a painful argument with a customer service rep.

I immediately sent a photo to the manufacturer and chairoffice customer service asking if they could supply me with the part that had broken and Mark in teknikoffice's customer service department replied almost immediately (appreciate this is around 10PM at night) and shipped me the entire back of one of the chairs as a replacement without any questions (which I got and fitted yesterday evening).

This is such good customer service it has made me want to talk about them.  Sometimes a faulty product can actually make you happier with the company if they deal with it correctly - and this is one of those cases; I don't think I have EVER encountered that good a customer service from any other company, so I would fully recommend if you're interested in a mesh back chair like an Aeron but a lot cheaper you get one from chairoffice.co.uk.

Permalink 

I even put OpenID on my own site in the latest iteration of its software as it is slowly growing in momentum, Scott Hanselman has details of how to implement OpenID support here.

This works particularly well with sites like Yahoo, where you can just put in the URL and be logged in near-instantly.

Permalink 

Very quick snippet that I just made use of to allow users to remotely change passwords over the web.  Just leave the domainname and username black to change the current users password.  This works great from ASP.NET and requires no special permissions, unlike some solutions that use LDAP or impersonation.

[DllImport("netapi32.dll", CharSet = CharSet.Unicode, 
   CallingConvention = CallingConvention.StdCall, SetLastError = true)]

static extern uint NetUserChangePassword(string domainname, string username, string oldpassword, string newpassword);
Permalink