Some Issues on Tagging Text

[Cameron Simpson]

First up, thank you for a well described problem! Remarks inline below.

On 24May2018 03:13, Subhabrata Banerjee <subhabangalore at gmail.com> wrote:
>I have a text as,
>
>"Hawaii volcano generates toxic gas plume called laze PAHOA: The eruption of Kilauea volcano in Hawaii sparked new safety warnings about toxic gas on the Big Island's southern coastline after lava began flowing into the ocean and setting off a chemical reaction. Lava haze is made of dense white clouds of steam, toxic gas and tiny shards of volcanic glass. Janet Babb, a geologist with the Hawaiian Volcano Observatory, says the plume "looks innocuous, but it's not." "Just like if you drop a glass on your kitchen floor, there's some large pieces and there are some very, very tiny pieces," Babb said. "These little tiny pieces are the ones that can get wafted up in that steam plume." Scientists call the glass Limu O Pele, or Pele's seaweed, named after the Hawaiian goddess of volcano and fire"
>
>and I want to see its tagged output as,
>
>"Hawaii/TAG volcano generates toxic gas plume called laze PAHOA/TAG: The eruption of Kilauea/TAG volcano/TAG in Hawaii/TAG sparked new safety warnings about toxic gas on the Big Island's southern coastline after lava began flowing into the ocean and setting off a chemical reaction. Lava haze is made of dense white clouds of steam, toxic gas and tiny shards of volcanic glass. Janet/TAG Babb/TAG, a geologist with the Hawaiian/TAG Volcano/TAG Observatory/TAG, says the plume "looks innocuous, but it's not." "Just like if you drop a glass on your kitchen floor, there's some large pieces and there are some very, very tiny pieces," Babb/TAG said. "These little tiny pieces are the ones that can get wafted up in that steam plume." Scientists call the glass Limu/TAG O/TAG Pele/TAG, or Pele's seaweed, named after the Hawaiian goddess of volcano and fire"
>
>To do this I generally try to take a list at the back end as,
>
>Hawaii
>PAHOA
>Kilauea
>volcano
>Janet
>Babb
>Hawaiian
>Volcano
>Observatory
>Babb
>Limu
>O
>Pele
>
>and do a simple code as follows,
>
>def tag_text():
>    corpus=open("/python27/volcanotxt.txt","r").read().split()
>    wordlist=open("/python27/taglist.txt","r").read().split()

You might want use this to compose "wordlist":

     wordlist=set(open("/python27/taglist.txt","r").read().split())

because it will make your "if word in wordlist" test O(1) instead of O(n), 
which will matter later if your wordlist grows.

>    list1=[]
>    for word in corpus:
>        if word in wordlist:
>            word_new=word+"/TAG"
>            list1.append(word_new)
>        else:
>            list1.append(word)
>    lst1=list1
>    tagged_text=" ".join(lst1)
>    print tagged_text
>
>get the results and hand repair unwanted tags Hawaiian/TAG goddess of volcano/TAG.
>I am looking for a better approach of coding so that I need not spend time on 
>hand repairing.

It isn't entirely clear to me why these two taggings are unwanted. Intuitively, 
they seem to be either because "Hawaiian goddess" is a compound term where you 
don't want "Hawaiian" to get a tag, or because "Hawaiian" has already received 
a tag earlier in the list. Or are there other criteria.

If you want to solve this problem with a programme you must first clearly 
define what makes an unwanted tag "unwanted".

For example, "Hawaiian" is an adjective, and therefore will always be part of a 
compound term.

Can you clarify what makes these taggings you mention "unwanted"?

Cheers,
Cameron Simpson <cs at cskk.id.au>