Not that I was using it for anything crypto-related, you understand, but the time-based thing does present a problem if you are using it in Scheduled Tasks. Particularly if you are using it to create unique, random temporary file names in two Scheduled Tasks that run at the same time.

Of course, this isn’t news. It’s an old new thing. But it was new to me and caused me to screw something up that I shouldn’t have and so on and so forth

Instead, we should pull random data from the system entropy, akin to *NIX’s /dev/random device. After doing some digging, it appears that what *NIX ‘stores’ in /dev/random, Windows ‘stores’ in the registry, a binary Seed value in the HKEY_LOCAL_MACHINE\Software\Microsoft\Cryptography\RNG key. Of course it does.

We can get to that in Windows batch files, with the REG.EXE command-line utility, usually found in the C:\Windows\system32 folder):

"%SYSTEMROOT%\system32\reg.exe" query HKLM\Software\Microsoft\Cryptography\RNG /v Seed

That gets us some version info and the binary data, in some nicely-formatted table, e.g…

! REG.EXE VERSION 3.0

HKEY_LOCAL_MACHINE\software\microsoft\cryptography\rng
    Seed        REG_BINARY      CC577B9D245DB08D34EC20C75B80CDF36822ACFB13F3FAD640F426948F5A05F1E9C956A8DB58F565CC6A1D2A947F93B38DA425D02D49BF30622FD8E7CFDBF63366A2357361C76ABA68720A04F28E7C4D

…which we can parse with FOR /f and convert to a similar 0-32767, like so:

FOR /f "usebackq tokens=3 skip=4" %%A IN (`"%SYSTEMROOT%\system32\reg.exe" query HKLM\Software\Microsoft\Cryptography\RNG /v Seed`) DO SET RND=%%A
SET /a RND=0x%RND:~0,4% %% 32768

The environment variable %RND% now contains a random number.

If you need a large quantity of random numbers, you may be concerned with repeated calls depleting the system entropy pool. This is a possibility, of course, but may be mitigated by keeping the last retrieved entropy data in an environment variable, extracting the first two bytes when you need to, removing those bytes from the data and refilling the entropy variable when it is empty. Something like this should do the trick

:get_random
IF NOT DEFINED ENTROPY FOR /f "usebackq tokens=3 skip=4" %%A IN (`"%SYSTEMROOT%\system32\reg.exe" query HKLM\Software\Microsoft\Cryptography\RNG /v Seed`) DO SET ENTROPY=%%A
SET /a RND=0x%ENTROPY:~0,4% %% 32768
SET ENTROPY=%ENTROPY:~4%
GOTO :EOF

Of course, this doesn’t seem to work on Windows Server 2008 R2 (surprise, surprise), as the HKLM...RNG\Seed value does not seem to exist in the registry (or at least cannot be read by REG.EXE), so I expect that means it doesn’t work in Vista or Win7, either. The closest alternative is the Seed value in the HKLM\System\RNG key, but that does not seem to update after reading, so is next to useless for this purpose.

The only alternative I have in Windows Server 2008 R2 is to invoke PowerShell (which is installed by default on our builds) to get a similar binary value, instead of REG.EXE. While you could instantiate an RNGCryptoServiceProvider object and call GetBytes on a suitably large byte array, it may just be sufficient to invoke Guid.NewGuid one or more times, instead (remembering to escape parentheses properly):

FOR /f "usebackq" %%A IN (`"%SYSTEMROOT%\System32\WindowsPowerShell\v1.0\powershell.exe" -Command [System.Guid]::NewGuid^(^).ToString^('N'^)`) DO SET RND=%%A

That said, if you have PowerShell, you might want to consider doing your script in that instead, if you can get past its awful, awful syntax…

That said, it did get me on to FileStream.  Looking through the Detail.txt file in the relevant dated subfolder of the C:\Program Files\Microsoft SQL Server\100\Setup Bootstrap\Log folder, I could see that the error was due to the uninstaller trying to remove permissions on the FileStream share that was created when I installed SQL Server 2008, originally.  From Administrative Tools >> Computer Management >> Shared Folders >> Shares, I tried to add the correct group permissions to the {INSTANCENAME} FileStream share, but it didn’t seem to make any difference to the uninstaller.

In the end, I got around it by editing the registry.  Opening regedit.exe from the Start >> Run window, I browsed to my instance’s registry branch, at HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\MSSQL10.{INSTANCENAME}\MSSQLServer and renamed the FileStream key to something arbitrary (I used “__FileStream“, but I expect you could use anything the installer wasn’t expecting).

Now when I ran the uninstaller, it couldn’t find the necessary registry branch, so assumed FileStream was uninstalled, skipped right over the step and completed successfully.  Then, I right-clicked the {INSTANCENAME} share in Computer Manager and clicked “Stop Sharing”.

I will post if I experience any issues with the desktop image, having performed this workaround.  In the meantime, hope this helps someone out there.

I’m used to doing things like ls -lrt | more to page the results of a directory listing, for example, but as a Windows user, I’m too used to being able to scroll my terminal (or command prompt) output to go back and forward and you just can’t do that in a pure terminal like I have been dealing with recently.

Enter the <(...) construct:

vi <(ls -lrt)

Now the output of ls -lrt is redirected to the vi editor as some kind of pseudo-file.  I can page up and down to my heart’s content.

I also discovered the du and comm utilities today (told you I was new) and using a combination of all of them can now compare the size of folders under two separate parents:

vi <(comm <(cd /dir1; du .) <(cd /dir2; du .))

This changes the current directory to /dir1, spools the size of all subfolders to a pseudo-file, changes the current directory to /dir2, spools the size of all subfolders to another pseudo-file, spools the comparison of the two pseudo-files to a third pseudo-file and shows that file in vi, for me to look around.

Perfect for eyeballing the results of a local rsync.  It took a whole batch file to do the same last time I tried in Windows.

However, deploying the finished scripts to a production database is a bit of a nightmare.  Essentially, if each script file represents a single database object, you have to ensure that you run them all in the correct order — especially if you have foreign key constraints (you do have foreign key constraints, don’t you?!).  Ordering all these scripts manually is a royal pain.

After the second or third time of being stuck in this situation, I decided enough is enough and set about writing a small app to do the job for me.  The results of this formed the mighty DependencyList.

The DependencyList is a class I have written that can be accessed and modified like a regular List<T> but also allows you to determine that certain items are dependent on each other.  During the enumeration process, using GetEnumerator or foreach, the dependencies of such items are always yielded first.

To assist with this (and promote possible extension to additional collections), I started out with a basic interface:

public interface IDependencyCollection<T> : ICollection<T>
{
    void AddDependencies(T item, params T[] dependsOnItems);
    void AddDependencies(int index, params int[] dependsOnIndices);
    void RemoveDependencies(T item, params T[] noLongerDependsOnItems);
    void RemoveDependencies(int index, params int[] noLongerDependsOnIndices);
    void ClearDependencies();
}

This determines that I will use basic ICollection principals, so the class is familiar to implementors but I have added the means to add and remove dependencies, based on value or index.

My basic means to determine dependencies is to declare a child class that provides a Yield method and contains a YieldingDelegate, which is a delegate with the same method signature as the Yield method.  I’ve chosen to make this a private child, with internal members, because it is not necessary for implementors to see/use this child class outside of the DependencyList context.  Laziness is responsible for my not wrapping my fields in properties and other such ‘good practice’ coding, but I figure that, since it is a private class, why add the overhead of a property get/set method call?

It is also important that an item is not yielded more than once per enumeration, so I have also added a private enum, which defines the three states of an item; not yielded, yielding and yielded.  The Yield process is only really concerned with ‘not yielded’ and ‘yielded’; ‘yielding’ state is primarily used in debugging to detect circualar dependencies:

public class DependencyList<T> : IDependencyCollection<T>, IList<T>
{
    private delegate IEnumerable<T> YielderDelegate();
    private enum YieldState { None = 0, Yielding, Yielded }

    private class DependencyListNode
    {
        internal readonly T value;
        internal DependencyListNode(T value) { this.value = value; }
        internal YielderDelegate Yielding;
        internal YieldState state = YieldState.None;
        internal void ClearYielding() { Yielding = null; }

        internal IEnumerable<T> Yield() { ... }
    }

    ...
}

The Yield method is responsible for checking the yield state and, if None, checking the YielderDelegate.  If the YielderDelegate is not null, it invokes the YielderDelegate.  After the YielderDelegate is finished, the Yield method yields its own value:

internal IEnumerable<T> Yield()
{
    if (state == YieldState.None)
    {
        state = YieldState.Yielding;

        if (Yielding != null)
        {
            IEnumerable<T> e = Yielding();
            foreach (T t in e) yield return t;
        }

        yield return value;

        state = YieldState.Yielded;
    }
}

My DependencyList class can now be fleshed out with the IList and IDependencyCollection interface implementations. As you will see I’ve not tried to reinvent the wheel as far as the basic List functionality goes; instead I add a List of DependencyListNodes to the class and expose it using the IList interface:

public class DependencyList<T> : IDependencyCollection<T>, IList<T>
{
    ...

    private readonly List<DependencyListNode> nodes = new List<DependencyListNode>();

    #region IList<T> members
    public void Insert(int index, T item)
    {
        nodes.Insert(index, new DependencyListNode(item));
    }
    public int IndexOf(T item)
    {
        for (int i = 0; i < nodes.Count; i++)
            if (Equals(nodes[i].value, item)) return i;

        return -1;
    }
    public void RemoveAt(int index) { nodes.RemoveAt(index); }
    ...
    #endregion

    #region ICollection<T> members
    public void Add(T item) { nodes.Add(new DependencyListNode(item)); }
    public void Clear() { nodes.Clear(); }
    public int Count { get { return nodes.Count; } }
    ...
    #endregion

    ...
}

(Note that I’ve not included the entire interface implementation here, but you should be able to get the idea.)

Next up is to add the IDependencyCollection implementation.  This is the key method behind how this class works.  Adding a dependency to a dependent item is a case of adding the Yield method of the item that must be yielded first, to the YielderDelegate of the dependent item.  Since the YielderDelegate is always invoked before the node’s value is yielded, the net effect is a recursion of Yield methods, in the order of dependency:

public class DependencyList<T> : IDependencyCollection<T>, IList<T>
{
    ...

    public void AddDependencies(int index, params int[] dependsOnIndexes)
    {
        if (dependsOnIndexes == null) throw new ArgumentNullException("dependsOnIndexes");
        if (index < 0 || index >= nodes.Count) throw new ArgumentOutOfRangeException(
            "index",
            index,
            "Index out of range"
        );
        foreach (int i in dependsOnIndexes)
            if (i < 0 || i >= nodes.Count) throw new ArgumentOutOfRangeException(
            "dependsOnIndexes",
            i,
            "Index out of range"
        );

        foreach (int i in dependsOnIndexes) nodes[index].Yielding += nodes[i].Yield;
    }

    ...
}

(Again, I’ve not fleshed out all methods here — RemoveDependencies(int, params int[]) is the same, except for the delegate assignment operator += becomes -=, and the ClearDependencies method invokes DependencyListNode.ClearYielding for all nodes, which sets the YielderDelegates back to null.  The value-based Add/Remove methods simply use IndexOf to find the first index of a value and then supplies said index to the Add/Remove methods shown).

The final step in the DependencyList is to implement the GetEnumerator() method.  This is similar to the Yield method of the DependencyListNode; the inner nodes List is, itself, enumerated and each node’s Yield method is called.  This causes the Yield/YielderDelegate recursion to occur for all nodes with dependencies.  Nodes without dependencies are simply yielded when they are enumerated in the nodes List.  I also ensure that only one enumerator can run at once, with a basic boolean check.  This is because I have to reset all the node states to YieldState.None, before I enumerate the nodes List:

public class DependencyList<T> : IDependencyCollection<T>, IList<T>
{
    ...

    public IEnumerator<T> GetEnumerator()
    {
        if (enumerating) throw new InvalidOperationException("Only one enumerator can run at a time");
        enumerating = true;

        foreach (DependencyListNode node in nodes) node.state = YieldState.None;

        for (int i = 0; i < nodes.Count; i++)
        {
            IEnumerable<T> e = nodes[i].Yield();
            foreach (T t in e) yield return t;
        }

        enumerating = false;
    }

    ...
}

Now my DependencyList class is complete, I can move on to the Database Project Deployer.

This consists of a basic form, with input folder and output file dialogue boxes.  For added functionality, I’ve also included a multi-select list box which is populated with all the SQL files found in the input folder.  The user can choose to include/exclude certain files using this list box:

The Database Project Deployer main window

When the user clicks the OK button, each selected file is added to a new class called the ScriptGenerator.  This class is responsible for parsing each SQL file, adding it to a DependencyList and determining if it has any dependencies on any other SQL file.  This class works on a single assumption that each SQL file corresponds to a given database object and that the name of the file is the name of the database object it creates, e.g. ‘table1.sql’.  It is not case sensitive.

The ScriptGenerator consists of a DependencyList<string>, which stores the contents of the SQL files and a Dictionary<string, int>, which maps the object name (taken from the filename) against its indexed position in the DependencyList.  It exposes two public methods, AddSqlFile(string) and Generate(string).

public class ScriptGenerator
{
    private readonly DependencyList<string> sqlList = new DependencyList<string>();
    private readonly Dictionary<string, int> nameIndexes = new Dictionary<string, int>();

    public void AddSqlFile(string sqlFilePath)
    {
        string fileName = Path.GetFileNameWithoutExtension(sqlFilePath).ToLowerInvariant();
        using (StreamReader reader = new StreamReader(sqlFilePath))
        {
            sqlList.Add(reader.ReadToEnd());
            nameIndexes.Add(fileName, sqlList.Count - 1);
        }
    }

    public void Generate(string outputFilePath)
    {
        ResolveDependencies();
        using (StreamWriter writer = new StreamWriter(outputFilePath))
        {
            foreach (string sql in sqlList)
            {
                writer.WriteLine(sql);
                writer.WriteLine();
                writer.Flush();
            }
        }
    }

    ...
}

The observant amongst you might have noticed the ResolveDependencies() method tucked into Generate.  This is the key part of the ordering system, as it is responsible for adding the dependencies within the DependencyList.  It does this simply by index-looping through DependencyList, using for and looping through the stored keys, trying to find that key within the SQL at that indexed position in the DependencyList.  There is an additional caveat to this process, namely that there are a number of different ways to write SQL files — the object names may be repeated and qualified by spaces, square brackets, quote marks, periods, etc.  To mitigate this, I have added a basic private struct to store the various qualifiers I have identified (there may be more) and looped through an array of them during each key check.  The private Qualifier struct looks much like this:

public class ScriptGenerator
{
    ...

    private struct Qualifier
    {
        internal static readonly Qualifier[] QUALIFIERS = new Qualifier[] {
            new Qualifier('"', '"'),
            new Qualifier('[', ']'),
            new Qualifier(' ', ' '),
            new Qualifier(' ', '('),
            new Qualifier(')', ' '),
            new Qualifier(')', '('),
            new Qualifier('.', '.'),
            new Qualifier('.', '('),
            new Qualifier('.', ' '),
            new Qualifier('(', ')')
        };

        private readonly char start;
        private readonly char end;

        internal Qualifier(char start, char end)
        {
            this.start = start;
            this.end = end;
        }

        internal string Qualify(string str)
        {
            return string.Concat(start, str, end);
        }
    }

    ...
}

The ResolveDependencies method looks like this:

public class ScriptGenerator
{
    ...
    private void ResolveDependencies()
    {
        for (int s = 0; s < sqlList.Count; s++)
        {
            string sql = sqlList[s]
                .ToLowerInvariant()
                .Replace(Environment.NewLine, "\n")
                .Replace('\t', ' ');

            foreach (string key in nameIndexes.Keys)
                foreach (Qualifier q in Qualifier.QUALIFIERS)
                    if (s != nameIndexes[key] && sql.Contains(q.Qualify(key)))
                    {
                        sqlList.AddDependencies(s, nameIndexes[key]);
                        break;
                    }
        }
    }
    ...
}

The end result is that all the SQL files are ordered appropriately and spooled to a single output script that can be executed on the target database with far less effort that all those separate scripts.  Obviously, your mileage may vary, depending on how you’ve written your SQL code, although most ordering errors can be resolved by adding suitable Qualifiers and remembering that it is just a dumb string.Contains(string) call that determines dependencies.  I’m sure there are better ways to parse object names from the SQL code, but this suits my needs in 99% of cases.

For those of you that don’t want to attempt to extrapolate all the missing code from above, I have zipped up the source and binary forms and make them available below for your convenience.  Note that I will not support the source or binaries and assume no liability for anything whatsoever.  For those of you with license concerns, consider the license as ‘use the app and DependencyList class however and wherever you like, but don’t sell it as a database script deployment tool’.  Does that seem fair?

• Database Project Deployer Binaries
• Database Project Deployer Source

Hope this helps with future Database Projects…

The OracleService<SID> service terminated unexpectedly.
It has done this <n> time(s).

For more information, see Help and Support Center at 

http://go.microsoft.com/fwlink/events.asp.

This was using some library code that we had developed and used successfully on numerous occasions, so it had us truly stumped.

After much soul-searching, one of my colleagues eventually narrowed it down to the length of the .NET project name. The project, when compiled, created a EXE file that had a really long name, which caused Oracle to shit bricks.

Wonderful. I hate how Oracle has such a problem with large object names.

Also, I’d love to know how an Oracle DBA would approach such a failure. How would it even be recorded? There are so many different log files in a standard Oracle installation that I really wonder where you might begin?

Examining the Application Event Log, I found the following error:

Event Type: ErrorEvent
Source: .NET Runtime 2.0 Error Reporting
Event Category: None
Event ID: 5000
Date: 26/09/2007
Time: 20:17:53
User: N/A
Computer: theDeploymentServer
Description:
EventType clr20r3, P1 theService.exe, P2 1.0.0.0, P3 46fa9f5d, P4 system.configuration, P5 2.0.0.0, P6 4333ae78, P7 1a2, P8 136, P9 ioibmurhynrxkw0zxkyrvfn0boyyufow, P10 NIL.
For more information, see Help and Support Center at
http://go.microsoft.com/fwlink/events.asp.

Not the most helpful of errors, but I could see the system.configuration entry, so I started with the theService.exe.config file, emptying the file of any meaningful data and adding it back in piece by piece until I recreated the error.

The error occurred on the following appSetting entry:

<add key="PluginsFolder" value="C:\Program Files (x86)\A&B\theService\" />

I realised that the appSetting contained an ampersand (&) witin its value, I realised that ampersand is a reserved character in XML and so the .NET configuration parser couldn’t validate the .config file.

To resolve this I replaced the ampersand with its XML-compliant counterpart (&amp;), so the key now read:

<add key="PluginsFolder" value="C:\Program Files (x86)\A&amp;B Company\theService\" />

Granted this looks a little weird but it solved the problem!

If you do not implement a method for VirtualPathProvider.GetFileHash(...), calls to this method will return null/Nothing.

If this happens, ASP.NET’s build/compile system doesn’t remember that it has already compiled the file and attempts to do so again. After 15 compiles, the AppDomain is unloaded and the application restarts. This can cause major irritation, particularly with performance, as the recompiles take time.

To get around it in my app, I am overriding GetFileHash to return a string concatenation of the VirtualFile’s name and Last Modified Date. If anyone knows of a better hash, then please let me know.

Having messed about with the options on the Login Screen, I inadvertently configured the X Server to try to login to a XDMCP server, by setting the startup server to Chooser.

When I rebooted, I couldn’t find any XDMCP servers, so I couldn’t login. I thought I’d try booting to a terminal mode, only to find that the kboot loader loaded Fedora if I left the keyboard idle for too long (about three seconds0).

Eventually I found out that I could get to a terminal from the XDMCP screen by pressing CTRL+ALT+F1. After that, I could log in as root and edit the GDM config file with VIM:
vi /etc/gdm/custom.conf
allowed me to edit the file, so I scrolled down to the [servers] section and changed the entry from:

[servers]
0=Chooser

to:

[servers]
0=Standard

After rebooting, the regular login screen was displayed.

I’ve learned two things from this:

1. CTRL+ALT+F1 opens a terminal from the login screen
2. I’m not going to mess with the startup screen again (I don’t even know what an XDMCP server is, let alone why I might want one!)