Showing posts with label Frequent Errors. Show all posts
Showing posts with label Frequent Errors. Show all posts

03 February 2016

Don't use curl in Dockerfiles

Today I broke a production server because a docker image had missing files. How could this happen? The build was green, the app was tested, docker built the image without any errors.
When I looked at the Dockerfile I found some curl commands to download additional .jar files directly from the internet. They contained some specialized logging logic that was not used during the tests, but made the application fail in production. It turned out that those files no longer exist on the internet. And that's the reason for my provocative blog titel:
curl will exit with code 0 even if the server responds with a 404 Not Found status code, because from a protocol point of view everything was correct. In this case scripts and dockerfiles will not fail but silently ignore failed downloads. To be honest, there is a --fail option for curl, but it's not fail safe and you have to know it. That's the reason why I now prefer wget for downloading files in scripts. It will by default return a non zero exit code if the download fails, and therefore failing the script. If you need to pipe the downloaded file to stdout you can do it with wget -O - <url> too.

20 November 2012

Understand and Prevent Deadlocks


Can you explain a typical C# deadlock in a few words? Do you know the simple rules that help you to write deadlock free code? Yes? Then stop reading and do something more useful.

If several threads have read/write access to the same data it is often often necessary to limit access to only on thread. This can be done with C# lock statement. Only one thread can execute code that is protected by a lock statement and a lock object. It is important to understand that not the lock statement protects the code, but the object given as an argument to the lock statement. If you don't know how the lock statement works, please read the msdn documentation before continuing. Using a lock statement is better than directly using a Mutex or EventWaitHandle because it protects you from stale locks that can occur if you forget to release your lock when an exception happens.

A deadlock can occur only if you use more than one lock object and the locks are acquired by each thread in a different order. Look at the following sequence diagram:



There are two threads A and B and two resources X and Y. Each resource is protected by a lock object.
Thread A acquires a lock for Resource X and continues. Then Thread B acquires a lock for Y and continues. Now Thread A tries to acquire a lock for Y. But Y is already locked by Thread B. This means Thread A is blocked now and waits until Y is released. Meanwhile Thread B continues and now needs a lock for X. But X is already locked by Thread B. Now Thread A is waiting for Thread B and Thread B is waiting for Thread A both threads will wait forever. Deadlock!

The corresponding code could look like this.

public class Deadlock
{
    static readonly object X = new object();
    static readonly object Y = new object();
   
    public void ThreadA()
    {
        lock(X)
        {           
            lock(Y)
            {
                // do something
            }
        }
    }

    public void ThreadB()
    {
        lock(Y)
        {
            lock(X)
            {
                // do something
            }
        }
    }
}

Normally nobody will write code as above with obvious deadlocks. But look at the following code, which is deadlock free:


public class Deadlock
{
    static readonly object X = new object();
    static readonly object Y = new object();
    static object _resourceX;
    static object _resourceY;

    public object ResourceX
    {
        get { lock (X) return _resourceX; }
    }

    public object ResourceY
    {
        get
        {
            lock (Y)
            {
                return _resourceY ?? (_resourceY = "Y");
            }
        }
    }

    public void ThreadA()
    {
        Console.WriteLine(ResourceX);
    }

    public void ThreadB()
    {
        lock(Y)
        {
            _resourceY = "TEST";
            Console.WriteLine(ResourceX);
        }
    }
}


But after re-factoring the getter for ResourceX to this

get { lock (X) return _resourceX ?? ResourceY; }

you have the same deadlock as in the first code sample!

Deadlock prevention rules


  1. Don't use static fields. Without static fields there is no need for locks.
  2. Don't reinvent the wheel. Use thread safe data structures from System.Collections.Concurrent or System.Threading.Interlocked before pouring lock statements over your code.
  3. A lock statement must be short in code and time. The lock should last nanoseconds not milliseconds.
  4. Don't call external code inside a lock block. Try to move this code outside the lock block. Only the manipulation of known private data should be protected. You don't know if external code contains locks now or in future (think of refactoring).

If you are following these rules you have a good chance to never introduce a deadlock in your whole career.



26 June 2011

Exception Logging Antipatterns

Here are some logging antipatterns I have seen again and again in real life production code. If your application has one global exception handler, catching and logging should be done only in this central place. If you want to provide additional information throw a new exception and attach the original exception. I assume that the logging framework is capable of dumping an exception recursively, that means with all inner exceptions and their stacktraces.

Catch Log Throw
catch (Exception ex)
{
    _logger.WriteError(ex);
    throw;
}

No additionally info is added. The global exception handler will log this error anyway, therefore the logging is redundant and blows up your log. The correct solution is to not catch the exception at all.

Catch Log Throw Other

catch (Exception ex)
{
    _logger.WriteError(ex, "information");
    throw new InvalidOperationException("information"); // same information
}
Same as Catch Log Throw, but now you have two totally unrelated log entries. Solution: use the InnerException mechanism to create a new exception and don't log the old one:
throw new InvalidOperationException("information", ex);

Log Un-thrown Exceptions
catch (Exception ex)
{
    var myException = new MyException("information");
    _logger.WriteError(myException);
    throw myException;
}

In this case an un-thrown exception is logged. This could cause problems, because the exception is not fully initialized until it was thrown. For example the Stacktrace property would be null. Solution: don't log, just attach the original exception ex to MyException:
throw new MyException("information", ex);

Non Atomic Logging
catch (Exception ex)
{
    _logger.WriteError(ex.Message);
    _logger.WriteError("Some information");
    _logger.WriteError(ex);
    _logger.WriteError("More information");
}

Several log messages are created for one cause. In the log they appear unrelated and can be mixed with other log message. Solution: Combine the information into one atomic write to the logging system: _logger.WriteError(ex, "Some information and more information");

Expensive Log Messages
{
    [...] // some code
    _logger.WriteInformation(Helper.ReflectAllProperties(this));
}
This one is really dangerous for your performance. An expensive log message is generated all the time even if the logging system is configured to ignore it. If you have expensive message, put the generation into an if block side by side with the logging statement:
if (_logger.ShouldWrite(LogLevel.Information))
{
    // do expensive logging here
    _logger.WriteInformation(Helper.ReflectAllProperties(this));

}
 

10 February 2011

Removing the mime-type of files in Subversion with SvnQuery

If you add files to subversion they are associated with a mimetype. SvnQuery will only index text files, that means files without an svn:mime-type property or where the property is set to something like “text/*”. At work I wondered why I couldn’t find some words that I know must exist. It turned out that subversion marks files stored as UTF-8 files with BOM as binary, using svn:mime-type application/octet-stream. This forces the indexer to ignore the content of the file.

I used SvnQuery to find all files that are marked as binary, e.g. t:app* .js finds all javascript files and t:app* .cs finds all C# files. With the download button at the bottom of the results page I downloaded a text files with the results. Because svn propdel svn:mime-type [PATH] can work only on one file (it has no --targets option) I had to modify the text file to create a small batch script like this:

svn propdel svn:mime-type c:\workspaces\javascript\file1.js
svn propdel svn:mime-type c:\workspaces\javascript\file1.js
svn propdel svn:mime-type c:\workspaces\javascript\file1.js

After this change indexing worked again. I now run a daily query that ensures that no sources files are marked as binary.

23 January 2011

What means “Skipped” in the subversion merge log?

I have seen many people who tend to ignore the “Skipped” messages that are sometimes displayed when you do a merge with subversion:

skipped_merge_tortoise skipped_merge_cmd

The “Skipped” message means that the merge operations wants to create or modify a file but that file already exists in your working copy and is not under version control (added to the repository:unversioned_fileThis can easily happen if you do a forward merge from trunk to a brunch and revert your local changes to the working copy (maybe because some manual merges went wrong). The revert command will leave new files as orphaned files in the working copy.

Don’t ignore those warnings! If you commit a working copy with “Skipped” warnings and do a backward (reintegrate) merge from branch to trunk, subversion will delete the formerly skipped files from the trunk! As a rule, you should never ignore “Skipped” warnings and if they occur fix the root problem then repeat the merge on a clean working copy.

merge_deletes

02 May 2010

Always databind SelectedItem before ItemsSource

The order in which bindings are defined in Xaml are relevant. Look at this Example: I have a ComboBox and a ViewModel. The ItemsSource of the ComboBox is bound to the Repositories property and SelectedItem is bound to SelectedRepository.

<ComboBox ItemsSource="{Binding Repositories}"
          SelectedItem="{Binding SelectedRepository}" />

The constructor of the ViewModel initializes the Repositories with a non empty collection and sets the SelectedRepository to the first element.

public ViewModel()
{
    Repositories = new List<string> {"First", "Second", "Third"};
    SelectedRepository = Repositories[0];
}

Yet, immediately after starting, I got null reference exceptions from other databound properties that are referencing the SelectedRepository property! After a little debugging I found out that when assigning the ViewModel to the DataContext of the view, the Binding Engine assigns null to the SelectedRepository! If you change the declaration of Databinding everything works as expected:

<ComboBox SelectedItem="{SelectedRepository}"
          ItemsSource="Repositories" />
Conclusion: the order of databinding declaration matters!

06 October 2009

Don't forget to run aspnet_regiis and ServiceModelReg

Every few month I run into weird installation problems on IIS so that ASP.NET applications (including .svc WCF services and Silverlight applications) don't run with misleading error messages. Today I was running into this problem again. Seems that I hadn't the correct permissions to download a .xap file from the ClientBin silverlight folder. After fiddling around for half an hour or so I eventually remembered to run aspnet_regiis.exe -i to reregister the current .net framework. This solved the problem immediately.
Even if this tool is located in "%windir%\Microsoft.NET\Framework\v2.0.50727" it seems to reregister the newest installed framework (in my case .NET 3.5 and Silverlight).

A similar problem can occur for WCF services with the .svc extension. In this case you need to run ServiceModelReg.exe -i again. This tool is located in "%windir%\Microsoft.NET\Framework\v3.0\Windows Communication Foundation"