Answer

They are people who he considers do not work properly. Those who lives on their wits are those who somehow get by, mainly by making it up as they go along eg lecturers, losers and louts. He also describes travellers (gypsies) ‘Lots of folk live up lanes
With fires in a bucket’ and how simple their lives are but ‘they seem to like it’ so they don’t need to do work to survive.