Search This Blog

Loading...

Wednesday, October 16, 2013

My painful experience in dealing with tmdhosting


I read how horrible it is to deal with an incompetent webhosting provider. And I have my own horror story to share. It’s a story that I hope will serve as a wakeup call to those who are in hosting providing business: beef up your offering, improve the competency of your support and do not oversell.

The hosting provider that I have problem with is the tmdhosting. In Chinese, TMD stands for a popular rude word that English has no equivalent of. When I first signed up with this hosting provider, I had no idea that I would curse the name.

This is how the story got started. My company was looking for a competent web host to host a vtiger app for our internal CRM use. tmdhosting is very aggressive in advertising, it even set up dedicated vtiger hosting page just to give the impression to the world that it is the vtiger hosting expert.

And so, we manually migrated the data in our old CRM to vtiger. And the moment we started using it, we found that the vtiger app was slow. We found out that the reason for this problem was because we were using shared hosting and our CPU limits were breached.

tmdhosting suggested that I upgraded to a VPS hosting package, which I complied. Before I could finish upgrading, another support personnel suggested me to take a dedicated server. I didn't take that because I couldn’t foresee why do I need a dedicated server. I reasoned that if a VPS couldn’t work out, then, and only then I would go for a dedicated server.

And so tmdhosting began a round of migration from the old shared hosting account to a VPS.

I thought that after the migration would increase the snappiness of my websites. How wrong I was. In fact, the performance of VPS was actually a few magnitude worse than shared hosting. I then filed a ticket to the support and asked the tmdhosting to fix the problem, there is no reason why VPS performance was worse than shared hosting performance.



See how useful the reply? I asked why VPS’s performance is worse than shared hosting, and the reply I got was “you need more resources”. Complete nonsense!

I then reiterated my questions, and again, the support ignored my questions and it appears as if the support was just reading from the scripts:

As you can see, all the tmdhosting staff was interested in, was to upsell me dedicated server hosting package, regardless of whether I needed it or not. tmdhosting was really putting their own interest above customer's.

After a few more exchanges, I decided to take the matters into my own hands and ssh into my server and did a few investigation. I narrowed the performance problem down to a disk issue:


51 seconds to transfer 1KB! This is simply outrageous. Now, of course I understand that for VPS, the CPU and RAM are guaranteed and the IO performance is not, and one bad container on a hardware node can drastically hurts the performance of other containers on the same node. But no matter what, such a slow IO performance is completely unacceptable.

I was very sure that I have a case against the tmdhosting provider, so I showed them the statistic and they bombed back with.... the talks of dedicated server again:





I was amazed and stupefied by the kind of response offered by tmdhosting. Here it is, a very simple and significant case that demonstrates your incompetency, and all you do, instead of fixing your failing, you upsell us another more premium package.

I then asked for ssh credentials to test the disk performance on my old shared hosting account, just to compare the disk IO operation characteristics, but this time tmdhosting refused to let me do so on the pretext that this may result in severe server overloads.




tmdhosting conveniently forgot that the very reason why I wanted to test the disk characteristics, was because their VPS performance was simply very bad. And since when running SSH command required government issued document? All these actions were meant to irritate me into giving up doing something that I deem necessary and which would further reveal their poor plans.

On top of the support incompetence, all the tmdhosting wants, is to sell me the expensive dedicated server package. But would an expensive dedicated server package solve my problem? I don’t know, and tmdhosting didn't care. All it cares is to upsell me expensive package.

You might say that I should just disengage them the very first moment they showed their incompetence but I didn’t. Yeah, serve me right.

Saturday, October 6, 2012

My experience with outsourcing a development project to India

Introduction

My company was looking at using vtiger in order to replace the old, aging in house CRM system. Our old CRM system is consisted of separated subsystems that are functioning on their own with absolutely no integration among themselves. Due to that the subsytems were developed in a haste and has never since been upgraded, it became harder and harder for us to maintain it.

Hence we have decided to use vtiger. We found that vtiger fulfills most of our needs. The module architecture of vtiger means that we can always write a third party module to provide extra functionality we need without touching the vtiger core source code.

And so, there went my quest to write a dongle module to supplement existing vtiger CRM functionalities. I have very limited PHP knowledge, and zero knowledge in the internal working of vtiger module architecture. So I wrote my specs, and went to freelancer.com to find someone who can do the job for me with a reasonable price.

To cut the long story short, my project did go on smoothly. Despite that many warning against the perils of outsourcing, my experience shows that outsourcing can still be cheaper than doing everything in house, with lower defect rates and faster completion time.

The Story:


  1. Before I started to even post the project on freelancer.com, I took great pains to compose a spec. I wrote the use case of the vtiger module, the UI specification, the reports I wanted to see and so on and so forth. The specification is so detailed that I specified the URL for each  forms, specified the elements, and the purpose of those elements for each forms and even designed the database schema for the backend. The point of this exercise was not to corner the implementer into doing the work my way, but rather to make sure that the implementer really understood my requirements.
  2. A lot of people hate writing specs; why you are writing something that no one reads, and something that is so ephemeral-- we all know requirements change with time, right? And if your spec is so complete, you might just as well code the whole thing yourself! 
  3. My reply to the above points are as follows:
    1. Regardless of how whimsical is the one who designated the requirements, changing the requirements in English is still significantly easier than changing the source code. 
    2. Designing the requirements in detail force you to think about difficult issues even before you start writing a single line of code. I suspect that those who refuse to nail down the requirements  are clueless about that they want, and they don't want to look stupid when the developers confront them. 
    3. Yes, requirements do change with time, but this is a point that for writing requirements, not against them. It's easier to document the requirement changes and their reasons in English, rather than in code, no?
    4. As to the point: "if your spec is so complete, you might just as well code the whole thing".  This is simply a straw man argument. I expect the implementer has some sense and experience in the domain and I don't expect him to ask me to specify in detail "does a button click works". Furthermore, coding is not just about UI, it has more than what meets the eye. Even if coding was just about the UI, it was still easier for me to "code" in English, rather than code in PHP or any other programming language. 
  4. There were a few bidders, and all their resume looked very impressive with multiple years of experience working with vtiger. But the resumes lie! You would need to interview them. And that was what I did; I interviewed them on their understanding of my spec and quizzed them on their vtiger ability. This was where the fun came in. Despite impressive resume, a few showed great difficulty in understanding the spec, another showed so much misunderstanding on vtiger that even I could detect, and another kept on saying my questions were elementary and he wasn't bother with them. All these kind of behavior didn't inspire confident. 
  5. But luckily I did manage to get a developer who showed mastery in vtiger and understanding of my spec. We hit off quickly. I had full confidence that the project was going to succeed. In fact, if you were not confident about the prospect of an outsourced project, don't continue!
  6. On the day of handover, we spent a few hours going through the applications, weeded out the bugs and enhanced a few features. Despite that the specs were completely written, and that the implementer showed good understanding during interview stage, when he delivered the thing to me, I still found a few parts that were not done properly. It took us a few real time sessions to hammer out all the issues, and we were done. 
  7. As predicted, the project was a resounding success!

My thoughts on engaging freelancers:

  1. Contrary to the prevailing opinion, there are well-qualified, smart, fluent in English knowledge workers in India, and their rates do come in cheap-- something like USD 20/hour. The only thing is you need to know how to identify them. I did this by interview them at length, ask them to show a sample of their past works, make sure that they really know their stuff,  and more importantly, they show deep understanding of my specs. If they cared to read about your specs and understood them, that meant that they were more likely to commit to your project and there was a higher chance of success.
  2. There will be a lot of "professional" firms that come knocking your door; they can show you impressive websites and resumes and they can even show you the past vtiger modules they did.  But you still have to take the pains to interview them. They might have done their projects but this is not a guarantee that they can do yours. 
  3. Indian slang can be a bit hard to follow for me as a Chinese. And I am sure that my slang can be a bit hard to follow for an European, or an African or anyone else. Which is why I make it a point that all correspondence must be done in writing form. Some would insist that a chat-meeting is more efficient and therefore more preferable. I would say that if you can't write down your requirements clearly, then perhaps you haven't really understood it enough. 
  4. The act of writing your specs down forces you to think deeply of what you want, and reduce the amount of time wasted on requirement clarification, communication and rework. No, this is not Waterfall; and even if it is, Waterfall model is good for this kind of projects!
  5. Set clear boundaries and expectations for your freelancer, and show empathy. You can do this by giving them good specs, make sure they really understand them, then give them free space to do their work, and only come back to them when the deadline looms near. 
  6. When dealing with a freelancer, make sure you structure your payment in stages. You pay only when a stage is done and you are satisfied with it. And you can easily cut your losses if you found that the freelancer was not performing up to the level you expect.
  7. Most importantly, go with your feeling! Before you hit off, make sure that you are comfortable with the guy you engage! 

Sunday, August 21, 2011

kindlemalaysia.com

It's a long hiatus from me. I haven't been updating this blog for ages, and now I am back, just to do some shameless promotion for my new side adventure.

To help my fellow Malaysians to get their hands on Kindle, I've decided to launch kindlemalaysia.com. Basically this is a website that allows you to purchase kindle the eBook reader direct, because the Amazon.com just won't ship its precious eBook reader device to Malaysia. So, anyone who is residing in Malaysia who wishes to get their hands on the device can go to the website and order one. The purchase process is painless, just like how would normally you buy things online.

If you have any questions you should read the FAQ, and contact admin@kindlemalaysia.com if your questions are not covered in FAQ.

One would think that my main purpose to setup this eCommerce shop is to make money. Nothing could be further from the truth. I have a nice full time job, and don't really need any part time income, thank you very much ( Although I would be much delighted if my friends who like to engage me as their wedding MC). And from my survey I highly doubt whether Malaysians are that much into book reading that I can make a nice living selling Kindle.

The reason I would do this, is to experiment with Google Adwords and online eCommerce. In other words, I am doing this for fun, and experience. Or maybe gaining a few talking points on this blog; writing about software isn't that fun after all. I hope that by one year later I can educate, regale everyone about my experience and what I've learnt.

We will see how it goes.

Saturday, August 7, 2010

CliSecure

One of the benefits ( or drawbacks) of .NET application is that the code can be easily decompiled by tools such as Reflector. This feature is tremendously useful if you have a third party application whose methods are  behaving bizarrely and you want to know what’s under the hood so that you can work around it. It is also comes in handy when you are trying to understand the algorithms offered by .Net framework ( such as LINQ’s Sort()) so that you can decide whether to use the existing one or writing your own.


However, for those who are in the line of selling software ( as opposed to, say open sourcing the software and then selling the service), the danger of having the whole application decompiled and resold by the competitors is just too great to ignore. Which is why .NET developers will have to obfuscate and encrypt their code before they package it for sale.

CliSecure .NET Obfuscator is a .NET code protection and licensing solution that I have been using for past one year. To date, we are very satisfied with the product and the service. Here are a list of the features that we really love
1.    .NET 4.0 support. We are in the process of migrating to .NET 4.0. The most important .NET 4.0 feature for us is PLINQ. For computationally intensive application, parallelism is the way to go as Moore’s Law is going to hit the wall. Instead of counting on the processor clock cycle to double up every 18 months, it is far more realistic to distribute computational loads to different cores to solve your speed issue. Which is why .NET 4.0 support is crucial for us in this matter. With CliSecure 5.2, we know that by the time we have transit to 4.0, we can release it straight to the customers without worrying about whether the protection tool is up to the task or not.
2.    Support various .Net Applications. The  .NET ecosystem is vast; Windows Form, ASP.NET, Silverlight, WPF, WCF, Windows Phone 7 etc. Different types of application development caters to different groups of needs, but they share the same CLR and framework. It is not unlikely that your application has to run on desktops, browsers and windows smartphones at the same time. The good thing about CliSecure is that it can handle all the nuances of the types, so that you don’t have to use different protection tools for different types of applications. One caveat though, some of the nonstandard .NET applications ( such as assemblies compiled by Matlab .NET builder and Silverfrost the fortran.Net compiler) may not be supported due to the nonstandard nature of those compiled assemblies. But in those cases, one can readily work around the limitation by editing the assemblies by hand. For me, this is not really a problem, as we seldom compile those nonstandard .NET code.
3.    Method Call Obfuscation. A lot of the protection tools keep the name of external calling method intact, because the names are needed in order to resolve the dependencies between different methods across different assemblies at runtime. This could create a security risk as hackers can determine what are the external calls your assemblies make and guess the content of your code. However, CliSecure solves this problem by replacing external calls with internal delegates so that the real, external calls are hidden from Reflector tools. With method call obfuscation, there is no problem in protecting the dynamic type objects, introduced in C# 4.0. So all your dynamic language fanatics! You can now have the privilege to write duck typing code that is previously restricted to dynamic type languages. But don’t blame anyone else if you introduce silly bugs that could have been caught by C# compilers. Greater flexibility comes with greater risks, I’m warning you. 
4.    64 bit support. Our application is a heavy number crunching engineering application. It uses up memory a lot. So 64 bit machine is the only hope for large projects. We were using other protection tools before stumbling upon CliSecure. Amazingly at that time ( about a year ago) there was not a single .NET obfuscator tool that protected a pure 64 bit application. When contacted on how to solve this problem, those tool providers advised us to compile our application as 32 bit app and run as 32 bit app on 64 bit OS. But this is completely impractical, as the very reason why we need 64 bit OS is because we need bigger memory.  I communicated this need to CliSecure and they got it done in one month. This was the deal breaker for us to use CliSecure.
5.    Stack Trace Translation. It’s often that when you are debugging a assembly which calls a protected assembly, you will get a gibberish stack trace when the crash happens. With the obfuscation, the real stack trace is also obfuscated and thus make debugging hard. However, with CliSecure 5.2, the stack trace is nicely preserved so that the developers won’t have to scratch his head and guess what’s going on wrong in the protected assembly. This would save us tremendous amount of time.
6.    Encryption, not just obfuscation. There is a difference between encryption and obfuscation. With obfuscation, Reflector can still see your method body and logic, with all the variables renamed, logic reshuffled ( not to the point of destroying the original code flow, of course). The weakness is that with enough determination, the hacker can still de-obfuscate the code and understand what is going on in your application. Encryption, on the other hand, completely hides the method body from hackers so there is no way of guessing what the code does beyond what is revealed from the method name.
7.    Compatible with reflection. One of the things I love about .NET is its rich meta data and the ability to use reflection to manipulate it. You can use reflection to dynamically infer an object’s type, create an instance of a type, access an object’s available methods, fields and properties. This is extremely useful when you are trying to  bind and display data in a declarative fashion. Unfortunately, obfuscation tools have the nasty habit of obfuscating private variables’ name, and make them inaccessible reflection, thus defeating the purpose of .NET rich metadata concept. CliSecure, on the other hand, doesn’t have problems with reflection. You can write the code in the way you want to without worrying about how the protection tool would work on it.
8.    Technical Support. The standard mantra of technical support in software world is “we’ll get back to you in 24 hours time”. But not everyone follows this. When we were researching for a .Net protection tool, there was one provider who got back to us only one week later, and his response was “sorry we can’t do this now, but we’ll keep you posted” and we never heard from him since. Providing good technical support is very important as we, the end users have our business to do; we can’t wait too long for our vendors to fix their problems or else our business would be affected. Provide timely and helpful technical support is sometimes more important than features. And for this reason I would normally eschew large software company with resellers and prefer the ones that are small and nimble enough to answer my question quickly. CliSecure .NET Obfuscator has never failed to resolve my queries in a timely manner. When I encountered bugs in their software, they would usually give me an update that fixed the problem within a reasonable time frame. The technical support should always factor into the consideration when purchasing a component.


Sunday, May 2, 2010

How to Use C# Client to Consume Google App Engine's rpc

Google App Engine allows one to run the web apps on Google infrastructure. This is especially enticing to developers because it takes care of hardware, hosting,  scaling, authentication and deployment issues.

I'm glad to learn that one can create desktop clients in any language to take to the Google App Engine backend. The way this can be done is as thus. First, get the excellent xml rpc python code from here, integrate it into your Google App Engine application.

The meat of your web service method is defined inside the application class, assuming that we have the following method in the application class:
def getName(self,meta, keyInput):                                                    
    return keyInput


This is how you can call this service method from C# client, by making use of the xml-rpc.net library:

[XmlRpcUrl("http://localhost:8080/xmlrpc")]
public interface AppTest: IXmlRpcProxy
{
    [XmlRpcMethod("app.getName")]
    string GetName(string number);

}

public string GetName(string keyInput)
{
        var  appProxy = XmlRpcProxyGen.Create<apptest>();
        appProxy.Url = "http://localhost:8080/xmlrpc/";
        return appProxy.GetName(keyInput);

}

So now your C# client can talk to Google App Engine backend with ease.

Isn't it simple? But it took me a few days to figure this out.

Thursday, February 18, 2010

5A99

I waited my friend at RHB bank opposite the Spring Shopping Mall, he was late, and the day was raining. This is not my exact idea of fun, I told myself. I was preparing to meet my ex-secondary schoolmates ( dubbed 5A99), meeting schoolmates can be fun, if the gathering can ignite the sense of kinship that bonded everyone in the first place. But it can be boring or even downright embarrassing if everyone is staring in (or past) everyone's presence, not knowing what to say. A bad weather and a-friend-late-on-time seemed like a bad omen for the thing to come. 

Finally, he came, so I followed his car to the destination. Kuching was moving up a lot since the days when I moved away from it. That place was a new restaurant that didn't exist ten years ago. As we parked our car and walked into the restaurant, I took a look at my watch. 7.20pm, 10 minutes to the dinner. Standing in front of the door was a handsome young man. He was Tze Siang, after years, he still looked as boyish as before. 

Suddenly, everyone seemed to appear at the same time, we enthusiastically exchanged handshakes, meeting old pals was good, but the atmosphere was a bit tense. We started to chit-chat, the conversation was a bit formal. And there were long pauses between conversation. Were we at lost of words to say to each other because of long-time-no-see? I wondered.

However, when we found a place to sit down, suddenly the atmosphere was heating up. First, a few crackers of chuckles, then, the voice volume rose, and crackers of chuckles gradually evolved into burst of laughters, from every corner of the room. 

We were exchanging our life-experience and jokes. We told one another of our experience from our walks of life, our friends listened attentively, sometimes added a few interesting remarks that set us all into laughters. The way we shouted across the table, it felt so much like the way when we shouted across the classroom. Our exaggerated facial expression when telling stories, those were the faces that we were when we were studying together. For a moment we traveled back in time, being our old gun-hoo self, resumed our innocent and carefree attitude.  There was never a moment in my life when I felt more 5A99 then now. 

The only difference was, the ladies were much more pretty and urbanized then ten years ago, and the men were more handsome, slickly dressed then before. 

To be sure, we are never the same again after started working. Our work and our commitment have taken a toll on us. We can no more make friends as heartfelt as we were ten years ago. There are times when we have to wear a mask when dealing with people. And sometimes I wonder whether earning money and recognition in exchange of innocence and genuine friendship is a good bargain.

But tonight's gathering reminded me of how precious and enduring the friendship forged since young is. We may only see each other once a year, or maybe less. And the time may come when we busied ourselves with our families, but deep inside my heart, I would never forget that at one point of time I studied together with 5A99, and tonight shall go down as one of the most memorable days in my life. Of course, we didn't befriend with each other because we wanted to get projects or benefits, that's the point I cherish most.

Thanks, 5A99. Love you all, always.




Tuesday, February 16, 2010

Face.com

Ever since I joined facebook I was overwhelmed by a large number of photos and photo albums created by my friends ( you know who you are :)). With a gazillion number of photos, photo browsing, finding and tagging became an addictive activity, and a consuming one.

For a long time I wished that facebook would  just import the picasa's ability to auto-recognize the faces and auto-tag them appropriately. I reckon that facial recognition technology shouldn't be so difficult that only Google can do it, and should be important enough for any photo-sharing-intensive sites to have it. Or else how can the users manually identify every single one of their friends in the photos?

So, I was thrilled to learn that face.com has finally taken the challenge to autotag the faces in facebook photos. The marketing material says that face.com employs "the most advance face recognition technology on earth". Although I can't verify the claim, but from my experience it is a very useful application that actually saves your time tremendously, unlike other pointless time-waster applications that suck away your time and serve absolutely no purpose at all ( I'm looking at you, SuperPoke! and MarketPlace). 

So, if you have facebook account, why don't you just head over to face.com, and lets it tags you and your friends photos. I'm sure you will find it as useful as I do.