I recently posted a find and replace regex for changing asserts into mex error messages. Here's a fancier way to do it without the find and replace.
#undef assert
#ifdef NDEBUG
# define assert( isOK ) ((void)0)
#else
# define assert( isOK ) ( (isOK) ? (void)0 : (void) mexErrMsgTxt(C_STR(__FILE__<<":"<<__LINE__<<": failed assertion `"<<#isOK<<"'"<<std::endl) ) )
#endif
Now any asserts will trigger a mex error and the message will be the same as a gcc assertion message. Source
Note: I updated this so that's it's not defined when NDEBUG is defined (just like the usual assert).