Hi,
I'm trying to reproduce the issue but it works as expected.
I get the error message "This value already exists in the list" (expected) and then when I hit Cancel the dialog closes (expected) and no blank records are inserted.
This is my scenario (same SP and build you have):
– custom list
– text field with no duplicates
– qListForm on list new page
– no custom actions but plain simple qListForm configuration
Is your scenario different than mine?