Musician

Let’s say there are two persons, Composer and Performer.

The Composer randomly selects three different Pitch which constructed by two part, Note and Octave. Note in the range of “A” to “G”, Octave in the range of “1” to “3”.

For example, here is a typical Pitch combined by ‘A’ and ‘1’, in which ‘A’ is the Note, and ‘1’ is the Octave.

Once Composer selected a Combination, Performer needs to guess it as quick as possible. After each guess, Performer get a feedback which indicates that:

  • how many pitches in the guess are included in the target (correct pitches)
  • how many pitches have the right note but the wrong octave (correct notes)
  • how many pitches have the right octave but the wrong note (correct octaves)

Now, the question is how to get the corrected answer as less times as possible.

--  Subject  : UniMelb 2019 SM1 COMP90048 Declarative Programming
--  File     : Proj1.hs
--  Author   : Mingzhe Du
--  Origin   : Mon Apr 8 2019 
--  Purpose  : This program for guessing a target chord. In each round of the game,
--             the program will generate a chord from a possible set, and then a feedback 
--             against the guess will be given. Depanding on these feedbacks, the aim of this
--             program is get the correct chord with as less as possible guess times.


module Proj1 (Pitch, GameState, toPitch, feedback, initialGuess, nextGuess) where

-- Pitch structure
data Pitch = Pitch { note :: Char, 
                     octave :: Char   
                   } deriving (Eq)
instance Show Pitch where
    show (Pitch note octave) = [note, octave]

-- Game State
data GameState = GameState { times :: Int,                  -- Guess times
                             cCombinations :: [[[Char]]]    -- Possible set
                           } deriving (Eq, Show)

-- Converting String to Pitch                          
toPitch :: String -> Maybe Pitch
toPitch (a:b:t)
    | not (null t) = Nothing
    | (elem note' ['A'..'G']) && (elem octave' ['1'..'3']) = Just Pitch {note = note', octave = octave'}
    | otherwise = Nothing
    where note' = a
          octave' = b

-- Comparing target chord and guessed chord
feedback :: [Pitch] -> [Pitch] -> (Int,Int,Int)
feedback pitch_a pitch_b
    | (length pitch_a == 3) && (length pitch_b == 3) = (c_p, c_n - c_p, c_o - c_p)
    | otherwise = (0,0,0)
    where get_key key = foldr (\x acc -> key x:acc) []
          c_p = getCounter pitch_a pitch_b 0                              -- Correct pitches
          c_n = getCounter (map note pitch_a) (map note pitch_b) 0        -- Correct notes
          c_o = getCounter (map octave pitch_a) (map octave pitch_b) 0    -- Correct Octaves

-- Initial guess
initialGuess :: ([Pitch], GameState)
initialGuess = (currentGuess, gameState)
    where currentGuess = combinationToPitch  (cGuess)
          gameState = GameState 0 all_combinations 
          all_items = getCombination "ABCDEFG" "123"
          cGuess:all_combinations = subsequencesOfSize 3 all_items        -- New guess and new possible set
          getCombination p_note p_octave = foldr (\x acc -> (map (\y -> y:[x]) p_note) ++ acc) [] p_octave
          combinationToPitch combinations = map (\(Just x) -> x) $ map toPitch combinations     -- Converting String to Pitch

-- Get the next guess
nextGuess :: ([Pitch], GameState) -> (Int,Int,Int) -> ([Pitch],GameState)
nextGuess (pGuess, pGameState) pFeedback = (cGuess, cGameState)
    where cGuess':cCombs = getNewCombination pGuess pCombinations pFeedback
          pCombinations = cCombinations pGameState
          cGuess = toChord cGuess'
          cGameState = GameState ((times pGameState) + 1) cCombs 
          toChord = map (\x -> Pitch (x !! 0) (x !! 1))

-- Help Functions 

-- remove an item from a list
removeItem :: (Eq a) => a -> [a] -> [a]
removeItem _ [] = []
removeItem x (y:ys) 
    | x == y = ys
    | otherwise = y : removeItem x ys 

-- get the number of same elements in two lists
getCounter :: (Eq a) => [a] -> [a] -> Int -> Int
getCounter [] y c = c
getCounter  (x:xs) y c
    | elem x y = getCounter xs (removeItem x y) (c+1)
    | otherwise = getCounter xs y c
 
-- Generate combinations by a specifc size    
subsequencesOfSize :: Int -> [a] -> [[a]]
subsequencesOfSize n xs = let l = length xs in if n>l then [] else subsequencesBySize xs !! (l-n)
    where subsequencesBySize [] = [[[]]]
          subsequencesBySize (x:xs) = let next = subsequencesBySize xs in zipWith (++) ([]:next) (map (map (x:)) next ++ [[]])

-- Converting a string list to pitch list
toChord :: [[Char]] -> [Pitch]
toChord a = map (\x -> Pitch (x !! 0) (x !! 1)) a 

-- retrive a new guess
getNewCombination :: [Pitch] -> [[[Char]]] -> (Int, Int, Int) -> [[[Char]]]
getNewCombination guess allCombinations pFeedback = foldr (\x acc -> if checkFeedback (toChord x) then  x:acc else acc) [] allCombinations
    where checkFeedback nChord = if pFeedback == (feedback guess nChord) then True else False

Haskell moment

Here are some Haskell code chunks, which both are simple recursion algorithms.

Quicksort is a sort of poster child for Haskell because everyone does it to showcase how elegant Haskell is.

quicksort :: (Ord a) => [a] -> [a]
quicksort [] = []
quicksort (x:xs) =
let smallerSorted = quicksort [a | a <- xs, a <= x]
    biggerSoted = quicksort [a | a <- xs, a > x]
in smallerSorted ++ [x] ++ biggerSoted
maximum' :: (Ord a) => [a] -> a
maximum' [] = error "List is empty!"
maximum' [x] = x
maximum' (x:xs) = (max x (maximum' xs))

replicate' :: Int  -> b -> [b]
replicate' n x
    | n <= 0 = []
    | otherwise = (x:(replicate' (n-1) x))

take' :: Int -> [a] -> [a]
take' n (x:xs)
    | n <= 0 = []
    | otherwise = x:(take' (n-1) xs)

reserve' :: [a] -> [a]
reserve' [] = []
reserve' (x:xs) = (reserve' xs) ++ [x]

BERT based sentence scenario detector

前两天用简单的多层感知器搭建了一个Word-level的detector模型。在模型的最后一次是用来Softmax,将Output Layer进行了分类。

对于场景识别这个问题,我目前先规定了可选的类别(比如Forest/ Ocean/ River/ College/ Suburb/ etc.)。这样一方面来说,可以简化detector的工作流程,另外也比较适应我们组目前的资源情况(识别场景之后需要提取事先准备好的Background,如果提取出了新的element也是无法获取到background resource的)。

上周我的想法是先使用Word embedding将Sentence转化为Sequence,然后使用Bi-LSTM或者直接使用Linear CRF对Sequence进行Sequence Tagging,以提取Sentence中涉及场景的Word。最后通过Word-level detector分析所选的Word,得到Sentence-level Scenario。

不过经过实验我发现,由于我手上只有不到500个短篇的儿童故事,还是没有标注的那种。就算我全部拿来进行标注,也只能生成不到5000个Phases。因为Labeler的资源比较紧张,我先用第一版的词表Detector模型生成了Labeling data,丢到CRF里面之后发现出了Person-entities,其他的类别基本无法有效识别出来。

于是这种方法暂时宣告失败。

周五晚上在公司发呆,突然觉得可以试一试力大砖飞的方法,直接使用Sentence-level embedding来作为Input。在这个模型里加入了CNN Layer,但其实单靠Dense Full connect Layer就已经可以在这个数据集上达到同样的效果了。

# 模型构建
model = Sequential([
Conv1D(filters=5, kernel_size=5, strides=1, padding='valid', input_shape=(768, 1), name="Convolution_Layer_1"),
AveragePooling1D(pool_size=5, strides=1, padding="valid", name="Pooling_Layer_1"),

Conv1D(filters=5, kernel_size=5, strides=1, padding='valid', name="Convolution_Layer_2"),
AveragePooling1D(pool_size=5, strides=1, padding="valid", name="Pooling_Layer_2"),

Flatten(name="Flatten_Layer"),

Dense(256, input_dim=3760, name="Dense_Layer_1"),
Activation('relu'),
Dropout(0.1),

Dense(32, input_dim=256, name="Dense_Layer_2"),
Activation('relu'),
Dropout(0.1),

Dense(11, input_dim=32, name="Dense_Layer_3"),
Activation('softmax'),
])


More than it

上周换了一副新眼镜。

由于用眼习惯不好,我一直都是高度近视。眼镜在我的生活里是必需品。

之前在澳洲眼镜架突然坏了,我找了整个City和Box Hill都没有找到可以修理或者更换的地方。只好自己从国内买了维修的零件和胶水自己修理。

为了把断在眼镜架里面的鼻托支架取出来,我用电钻给鼻托支架的支撑位钻了一个小洞,之后又往里面灌了不少解胶剂。估计是因为镜架是塑钢材质的,我修好之后发现镜架居然裂了一条缝,还好我已经用胶水填了缝。

结果上周的某一天不知道怎么回事,戴上眼镜之后就觉得很不舒服。仔细一看才发现是因为之前的那个缝隙裂开了,导致镜片松动,屈光位置对不上了。

想了想再用胶水粘起来也不是长久之计,如果回了墨尔本再断了,就不是这个容易修理的了。于是果断换了一副新的眼镜。

这次配了一个比较大的镜框,据说这样看东西会明亮许多,但估计也得需要时间去适应一下。这两天戴这副眼镜的话,还是有一点晕的。还有就是,老妈一直非常关心我的视力问题,因为她身边有不少同事都因为高度近视吃了不少苦头。我自己其实也深受其害,希望自己以后可以注意用眼,少看一点电脑屏幕,少刷一点手机。

有的时候,闭上眼睛,你可以看到更多。

通用场景识别器

今天是新年第一天上班,然后想到这周只用上三天班就很开心。

由于Sequence Tagging需要大量的标注数据,我这边暂时没有数据源,所以今天下午就先用现有的标注数据集做了一个场景Softmax分类器。

原理十分简单,使用Word2vec (之后可能会考虑换成BERT,但是这两天BERT在我这里表现还不是很理想,所以先用顺手的工具搭建一下Demo)生成词向量。之后通过标注数据集合,将词表里所有的词分成以下几个大类(类别可以由具体的使用场景确定,我这里的分类主要是为了适配童话故事的情况)。

# 模型构建
model = Sequential([
Dense(32, input_dim=200),
Activation('relu'),
Dropout(0.1),

Dense(16, input_dim=32),
Activation('relu'),

Dense(9, input_dim=32),
Activation('softmax'),
])

用Keras搭建了一个最简单的多层感知机,加上Dropout,开始喂数据。最后可以达到96%左右的Accuracy,算是基本可以使用了。

 test loss:  0.09276254528926478
test accuracy: 0.9666666666666667

现在这套模型已经可以识别任意词的场景类别了。下一步就是使用Sequence Tagging找出描述场景的位置了。

隐含语义分析

最近在做NLP的一些工作。

遇到的问题是:Precisely detect scenario/weather/time/etc in a paragraph-level document.

对于这个问题,我作为一个NLP外行,首先想到的思路是建立相应的词表,然后对每一个Sentence进行Tagging。在完成Coding的工作后,在测试中效果还是可以的(毕竟用Word2Vec聚类建立了好大一份词表)。但主要的问题是Overtrigger,这个模型无法区分那些地方才是真正描述Target的,更无法理解一词多义的情况。

于是在第一版模型可以基本Cover住下游业务需求之后,我Mentor对Precision有了进一步的要求(手动GG)。

在想了好几天之后(其实主要时间都用来划水),我想了一个不太成熟的方案,先记下来等元旦过完了回来再和Mentor讨论一下:

  • 使用TF-IDF-Based Sequenced List取代完整的Word Segment List。
  • 使用BERT取代Word2vec进行Word Embedding。
  • 尝试使用Latent Semantic Analysis,进行第一轮无监督学习,总而获取到Sentence-level的Tagging information(说到底,还不是因为没有Labeler)。
  • 使用SVD左矩阵反推Word-level Influence Factor,生成NER training data(这一步的话,为了保证模型质量,估计还是得请Labeler帮忙看一下。这个样子的话,比从什么都没有标注要轻松一些)。
  • 尝试双向LSTM训练隐含序列,或者直接通过BERT Pre Trained Model 产出Word Embedding(反正BERT的Embedding是自带Context的,啦啦啦)。
  • 使用线性链的条件随机场进行Sequence Tagging。这一步的话,参考目前的NER模型,在Location/Person/Organization上有95+的F1

如果这几个流程可以按照预期来进行的话,那效果应该还是不错的。

How to SUDO in Windows

You may be a senior user of Linux, and sudo is your lifestyle. However, how to use “sudo” in Windows except using right-click is still a problem for you probably.

Since I am working for Microsoft now, Windows operation system is the unique choice for me. Through a long and hard exploration, I finally figured out how to use “sudo” like in Linux in Windows prompt.

Here is the command for your reference:

runas /noprofile /user:Administrator "your command"

Then, You might be required to input your password like in Linux (U cannot see what you input too, so just typing…)

帝都十日

算算日子,来帝都已经有十天了。

不算工作的五天,大大小小的饭局约了有十多个。感觉减肥的大计又要暂时搁浅了。

之前在墨尔本报了健身房,疯狂跑步,基本上每天都可以轻一公斤的样子,最轻的时候甚至差点就要突破90大关了。但是最近各种吃吃吃,导致体重大有回升到一百的趋势。

然后工作其实还蛮轻松的。感觉整个小组是一个大的Scrum,然后我是一个小的Scrum team。每天只需要和Mentor沟通一下对好需求就可以了。

至于剩下的时间的话,就可以自由支配了。所以其实可以自己学到很多东西。虽说外面一直盛传微软不用加班,但是好歹要入乡随俗,STCA还是多多少少要加一点班的,不过范围全由自己掌握。

本来说上个周末要去圆明园和颐和园,但是时间总是不够用,和朋友玩了一会儿就没有时间去逛园子了。

周一刷朋友圈的时候看到Han也来了北京,原因竟然只是为了看升国旗,果然是思想觉悟超级高的党员。然后就愉快的和Han约了周四晚上的局。这应该也是我们第一次在国内见面吃饭。

上个周末和晓波约了周末去滑雪,感觉应该会很好玩,美滋滋。所以估计周末的另外一天我会抽出来去逛圆明园。

上周和这周的情况差不多就是这个样子了,希望假期在微软可以过的开心吧,顺便多学一点东西。

Cheers

Start Windows

这两天开始认真工作了。

由于代码仓库的权限一直没有审批下来,我昨天划了一天的水。

可能是因为巨硬的体量实在是太大了,各种流程的审批都有点慢。而且不知道问什么,感觉Check-in的引导工作也不是非常理想,很多东西都需要自己打电话去咨询,这无疑增加了沟通成本。

在其他方面,微软的东西做的还是可以的(除了经常性的exception之外)。就操作系统来说,在微软肯定是得用Windows的系统了,但是Windows真的是对开发人员不太友好,Shell好难用。

于是我默默的搜索了一波Windows效率工具… 希望可以稍微缓解一下现在尴尬的情况…

今天和豪翔约了午饭,希望可以请教前辈大佬的经验吧。