For me, the dimensions are key. Also, I am a big fan of both JP's sets and Phil Dickinson's. They are made by competitive kubb players that play in a lot of tournaments and it shows in their quality of work.
Features that I like are the rounded corners on the kubbs and rounded ends of the batons. I feel this enhances game play and extends the life of the equipment.
JP's poplar sets are the standard for most tournaments, so I think it is important to practice with one of his sets for competitive minded players. Phil's are starting to be used in more tournaments. They are great and will last a long time as they are hardwood for the premium sets. He also make a quality poplar set that would work well if you are practicing for a tournament that uses poplar.
Both are great guys and huge parts of the kubb community. I have to disagree with Jesse on one thing... Phil is undoubtedly the biggest Kubb fanatic there is. :)