Somehow it was hard to find this on the internet.
Example of adding a custom http handler to undertow in EAP 7 through CLI.
/subsystem=undertow/configuration=filter/custom-filter=my-useful-name-for-filter:add(class-name=…,module=…)
/subsystem=undertow/server=default-server/host=default-host/filter-ref=my-useful-name-for-filter