Write Three Labeling Functions and Analyse Their Conflicts

For the support-message problem write three labeling functions (LFs): one for billing, one for refund and one for login. On a 200-item gold dev set the counts are as follows. The invoice LF (billing) fires on 33 items, 22 of them truly billing. The refund LF (refund) fires on 37 items, 11 of them truly refund. The password LF (login) fires on 28 items, 21 of them truly login. The invoice and refund LFs both fire on 12 items; the refund and password LFs both fire on 5 items; the invoice and password LFs never fire together.

1. Give each LF as a rule, and compute its coverage, precision and conflict rate on the dev set.
2. For an item on which only the invoice and refund LFs fire, the label model has estimated that the invoice LF fires on 50 percent of billing and 10 percent of refund items, and the refund LF on 10 percent of billing and 60 percent of refund items. With class priors 0.28 (billing) and 0.10 (refund), compute the posterior, ignoring all other classes, and compare it with majority vote.
3. Someone adds three near-copies of the refund LF. Recompute the posterior for an item on which the invoice LF and all four refund LFs fire, and explain what went wrong and how to fix it.
4. Say what you would do about the weak refund LF.