Mturk seems like a great way to create datasets as well, there are a lot of stories of people creating datasets with mturk jobs however you will need some type of algorithm for filtering out people who are just clicking to get through things and not really labeling anything.

> Machine learning only learns how to categorize things into predetermined categories.

This is just one type of machine learning called classification, there are others like regression and clustering which can be combined to create more robust models. Look at the technology behind Cylance's product which identifies files as malicious or not pre-execution. They are not just using classification.

Yes you can, that's part of the appeal of applying machine learning to security. They don't rely on things like signatures or existing heuristics to identify things as malicious.