RSS

Tag Archives: Culture

Culture Issues on ASP.Net Websites

When building a website that can display the pages in the language and culture of the user, there are some type conversion issues that developers need to be aware of. When building our BlogTalkRadio.com site we hit a few of them and have learned some good lessons.

Over the years, our BlogTalkRadio.com web site has seen a number of strange errors littering our exception logs. Often times we see errors that we cannot reproduce ourselves. When analyzing these errors we sometimes find a correlation whereby every user seeing the error has the same language culture which is different than our’s here in the US. (en-US).

Places we have seen issues:

  1. Number Conversions
  2. DateTime Conversions
  3. String Comparisons

One way to combat these issues is to specify and control the culture being used. In Asp.net, many developers do not consider that when somebody comes to your site, and .Net framework executes your code, it will use the culture of the browser visiting the site to decide how to process the code.

Additionally, designers often don’t take into account how another culture’s date and numeric format will look on a page. To say nothing of how language and special characters cause issues. JSON is notoriously tied to US culture. We have seen many issues in JSON and XML with EBCIDIC characters causing errors and issues. Encoding these characters are a MUST.

CultureInvariant is a good tool used for converting from string to Datetime or number and for comparisons so it can handle a number formatted as 1.2 and Russian format such as 1,2 seemlessly. (http://stackoverflow.com/questions/2423377/what-is-the-invariant-culture)

Here are some common places where not thinking about Culture can come back to bite you. Below I default to U.S. culture when CultureInvariant wont do.

  • Some other Cultures such as Saudi Arabia have a different calendar. Therefore creating a date from year, month and date will have unexpected results. The year 2012 has not happened in their calendar yet so will error.

    Error Prone:

    showDate = new DateTime(year, month, day)

    Better:

    var CI = System.Globalization.CultureInfo.GetCultureInfo("en-US");
    showDate = new DateTime(year, month, day, CI.Calendar)
  • Calendar is also an issue going the other way. We had an issue here because urls were getting generated with completely incorrect dates in cultures with a different calendar

    Error Prone:

    var dt = ActualShowDate!=DateTime.MinValue?ActualShowDate :ShowDate;          
    sb.Append("/");
    sb.Append(dt.ToString("yyyy”));
    sb.Append("/");
    sb.Append(dt.ToString("MM”));
    sb.Append("/");
    sb.Append(dt.ToString("dd”));
    sb.Append("/");

    Better:

    var CI = System.Globalization.CultureInfo.GetCultureInfo("en-US");
    sb.Append("/");
    sb.Append(dt.ToString("yyyy", CI.DateTimeFormat ));
    sb.Append("/");
    sb.Append(dt.ToString("MM", CI.DateTimeFormat));
    sb.Append("/");
    sb.Append(dt.ToString("dd", CI.DateTimeFormat));
    sb.Append("/");
  • Below is another example of where a different calendat can generate bad dates. This started to bite us the most when we started using page caching. If a Saudi Arabian visitor hit the page and caused it to be cached then all visitors for the duration of the cache would get the strange dates.

    Error Prone:

    hlArchiveURL = _baseURL + archive.MonthStarting.ToString("yyyy/MM”);

    Better:

    hlArchiveURL = _baseURL + archive.MonthStarting.ToString("yyyy/MM", new CultureInfo("en-us", false));
  • We have have found that the decimal version numbers in the UserAgent are always in the U.S. format. Except of course when there is a letter appended such as IE 7.0b. So the Error Prone code was erroring for non US users since 7.0 is not valid in Brazil.Error Prone:
    if (Request.Browser.Browser == "IE"){
        double browVer = Convert.ToDouble(Request.Browser.Version);
        if (browVer <= 7.0) { 
        	// put hacks for ie7 here.
        }
    }

    Better:

    var CI = System.Globalization.CultureInfo.GetCultureInfo("en-US");
    if (Request.Browser.Browser == "IE"){
        double browVer = Convert.ToDouble(Request.Browser.Version, CI);
        if (browVer <= 7.0) { 
        // put hacks for ie7 here.
        }
    }
  • We use Elasticsearch for our searches. When we tweaked the ranking alghoritm to use decimal values, all of a sudden people outside the US were getting errors when trying to search.This is the call within our code to the ElasticSearch client.
    booleanQuery.Should(new TermQuery("HostDisplayName", term,  1.3, true));

    Within the ElasticSearch Client code this line was giving us trouble because the JSON format when it hit the server wasn’t supported. The server could not handle a number in the format 1,3. Without specifying culture, many non US users were generating 1,3 in the JSON vs 1.3.

    Error Prone:

    writer.WriteRawValue(string.Format("{{\"filter\": {{\"term\": {{ \"{0}\" : \"{1}\" }} }}, \"boost\": \"{2}\" }}", term.Field, term.Value, term.Boost.ToString()));

    Better:

    writer.WriteRawValue(string.Format("{{\"filter\": {{\"term\": {{ \"{0}\" : \"{1}\" }} }}, \"boost\": \"{2}\" }}", term.Field, term.Value, term.Boost.ToString(Utils.JsonBuilder.UsCultureProvider)));
  • Different languages have different comparison rules. Some languages have two character letters, others have an accent mark or tilde over a letter that makes it different. We learned the hard way that in Turkish culture I != i. In fact we discovered that ToLower(“I”) != “i”. Capital I is not the same letter as lowercase I. Therefore, doing a ToLower didn’t help. We were not getting a string match when someone came from Turkey. Therefore we switched to use ToLowerInvariant() which did the trick.

    Error Prone:

    where _includeExtensions.Contains(Path.GetExtension(file)) && !file.ToLower().Contains("includefilesource\\")

    Better:

    where _includeExtensions.Contains(Path.GetExtension(file)) && !file.ToLowerInvariant().Contains("includefilesource\\")

    Please note that for brevity I did not include try/catch logic to account for bad data with some of the code.

HOWEVER,

There is one setting that can be made in the web.config that would have made all of these issues moot.

    <globalization culture="en-US" uiCulture="en-US" />

This setting sets the entire site to US culture and does not take the client’s culture into account. This can be overridden by setting a particular culture within the code on a case by case basis if needed.

The downside of this setting is that you cannot use .Net’s functionality to display dates, numbers and times in the local user’s format. For our BlogTalkRadio.com site, we don’t want to use this functionality anyway.

For high volume sites, caching pages and output caching is common. If someone from an Arabic culture hits the page and causes it to be cached and then someone in New York hits the cached page, the NY will be a bit confused by the odd Arabic date that they will see. Therefore, any client formatting should be done via JS on the page itself.

Summary: It is important to think about how your website will handle different cultures in your application. Not thinking about it will cause errors and a bad experience for most users outside of your home country. You can handle it via global settings or detailed individual settings.

 

Tags: , ,