Removing a list of characters in string

Go To


I want to remove characters in a string in python:

string.replace(',', '').replace("!", '').replace(":", '').replace(";", '')...

But I have many characters I have to remove. I thought about a list

list = [',', '!', '.', ';'...]

But how can I use the list to replace the characters in the string?

2012-04-04 18:28
by Laura
See for various solutions and a nice comparison - Martijn de Milliano 2012-04-04 18:32
It's a pity that Python (which is said to come with batteries included) does not handle this use case out of the box. PHP's function str_replace does it - you can pass an array as the first argument and a string as the second ( ) - JustAC0der 2017-01-13 23:09


If you're using python2 and your inputs are strings (not unicodes), the absolutely best method is str.translate:

>>> chars_to_remove = ['.', '!', '?']
>>> subj = 'A.B!C?'
>>> subj.translate(None, ''.join(chars_to_remove))

Otherwise, there are following options to consider:

A. Iterate the subject char by char, omit unwanted characters and join the resulting list:

>>> sc = set(chars_to_remove)
>>> ''.join([c for c in subj if c not in sc])

(Note that the generator version ''.join(c for c ...) will be less efficient).

B. Create a regular expression on the fly and re.sub with an empty string:

>>> import re
>>> rx = '[' + re.escape(''.join(chars_to_remove)) + ']'
>>> re.sub(rx, '', subj)

(re.escape ensures that characters like ^ or ] won't break the regular expression).

C. Use the mapping variant of translate:

>>> chars_to_remove = [u'δ', u'Γ', u'ж']
>>> subj = u'AжBδCΓ'
>>> dd = {ord(c):None for c in chars_to_remove}
>>> subj.translate(dd)

Full testing code and timings:


import re

def remove_chars_iter(subj, chars):
    sc = set(chars)
    return ''.join([c for c in subj if c not in sc])

def remove_chars_re(subj, chars):
    return re.sub('[' + re.escape(''.join(chars)) + ']', '', subj)

def remove_chars_re_unicode(subj, chars):
    return re.sub(u'(?u)[' + re.escape(''.join(chars)) + ']', '', subj)

def remove_chars_translate_bytes(subj, chars):
    return subj.translate(None, ''.join(chars))

def remove_chars_translate_unicode(subj, chars):
    d = {ord(c):None for c in chars}
    return subj.translate(d)

import timeit, sys

def profile(f):
    assert f(subj, chars_to_remove) == test
    t = timeit.timeit(lambda: f(subj, chars_to_remove), number=1000)
    print ('{0:.3f} {1}'.format(t, f.__name__))

print (sys.version)
PYTHON2 = sys.version_info[0] == 2

print ('\n"plain" string:\n')

chars_to_remove = ['.', '!', '?']
subj = 'A.B!C?' * 1000
test = 'ABC' * 1000



print ('\nunicode string:\n')

    chars_to_remove = [u'δ', u'Γ', u'ж']
    subj = u'AжBδCΓ'
    chars_to_remove = ['δ', 'Γ', 'ж']
    subj = 'AжBδCΓ'

subj = subj * 1000
test = 'ABC' * 1000





2.7.5 (default, Mar  9 2014, 22:15:05) 
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.0.68)]

"plain" string:

0.637 remove_chars_iter
0.649 remove_chars_re
0.010 remove_chars_translate_bytes

unicode string:

0.866 remove_chars_iter
0.680 remove_chars_re_unicode
1.373 remove_chars_translate_unicode


3.4.2 (v3.4.2:ab2c023a9432, Oct  5 2014, 20:42:22) 
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]

"plain" string:

0.512 remove_chars_iter
0.574 remove_chars_re
0.765 remove_chars_translate_unicode

unicode string:

0.817 remove_chars_iter
0.686 remove_chars_re
0.876 remove_chars_translate_unicode

(As a side note, the figure for remove_chars_translate_bytes might give us a clue why the industry was reluctant to adopt Unicode for such a long time).

2012-04-04 18:29
by georg
The second method raises an error TypeError: translate() takes exactly one argument (2 given). Apparently it takes dict as an argument - antonavy 2014-09-18 13:02
@antonavy - the 2nd solution does work - but only of the string is not unicode (for which a different translate() is needed - FuzzyAmi 2014-12-02 13:50


You can use str.translate():

s.translate(None, ",!.;")


>>> s = "asjo,fdjk;djaso,oio!kod.kjods;dkps"
>>> s.translate(None, ",!.;")
2012-04-04 18:31
by Sven Marnach
@thg435: Nobody asked for this, but anyway: s.translate(dict.fromkeys(map(ord, u",!.;")))Sven Marnach 2012-04-04 18:43
This (and @PraveenGollakota's) simultaneous answer is exactly what @Laura asked for and should be the preferred answer(s) - hobs 2014-01-15 07:07
why python3:TypeError: translate() takes exactly one argument (2 given - Gank 2015-09-19 10:27
@Gank: The unicode.translate() method has different parameters than the str.translate() method. Use the variant in the above comment for Unicode objects - Sven Marnach 2015-09-19 13:03
@SvenMarnach what is map(ord, u",!.;"))? and does u stand for unicode - Jun711 2018-08-23 00:36
@Jun This maps the ord() function on all characters of the Unicode string u",!.;", resulting in a list (Python 2) or an iterator (Python 3) of Unicode code points. The u denotes a Unicode string in Python 2. In Python 3 it's optional, since strings are Unicode strings by default (and early versions of Python 3 did not even support the u"" syntax) - Sven Marnach 2018-08-23 08:07


You can use the translate method.

s.translate(None, '!.;,')
2012-04-04 18:31
by Praveen Gollakota


''.join(c for c in myString if not c in badTokens)
2012-04-04 18:53
by ninjagecko
Useful in similar cases not based on chars and strings + - Wolf 2014-09-20 07:59


If you are using python3 and looking for the translate solution - the function was changed and now takes 1 parameter instead of 2.

That parameter is a table (can be dictionary) where each key is the Unicode ordinal (int) of the character to find and the value is the replacement (can be either a Unicode ordinal or a string to map the key to).

Here is a usage example:

>>> list = [',', '!', '.', ';']
>>> s = "This is, my! str,ing."
>>> s.translate({ord(x): '' for x in list})
'This is my string'
2016-09-11 12:33
by Dekel


Another approach using regex:

''.join(re.split(r'[.;!?,]', s))
2012-04-04 18:51
by alan


you could use something like this

def replace_all(text, dic):
  for i, j in dic.iteritems():
    text = text.replace(i, j)
  return text

This code is not my own and comes from here its a great article and dicusses in depth doing this

2012-04-04 18:31
by krystan honour


Why not a simple loop?

for i in replace_list:
    string = string.replace(i, '')

Also, avoid naming lists 'list'. It overrides the built-in function list.

2013-10-23 07:47
by aIKid


Also an interesting topic on removal UTF-8 accent form a string converting char to their standard non-accentuated char:

What is the best way to remove accents in a python unicode string?

code extract from the topic:

import unicodedata

def remove_accents(input_str):
    nkfd_form = unicodedata.normalize('NFKD', input_str)
    return u"".join([c for c in nkfd_form if not unicodedata.combining(c)])
2014-05-31 07:23
by Sylvain


Perhaps a more modern and functional way to achieve what you wish:

>>> subj = 'A.B!C?'
>>> list = set([',', '!', '.', ';', '?'])
>>> filter(lambda x: x not in list, subj)

please note that for this particular purpose it's quite an overkill, but once you need more complex conditions, filter comes handy

2015-03-24 17:20
by rioted
Also note that this can just as easily be done with list comprehensions, which is way more pythonic in my opinion - rioted 2016-11-15 11:47


simple way,

import re
str = 'this is string !    >><< (foo---> bar) @-tuna-#   sandwich-%-is-$-* good'

// condense multiple empty spaces into 1
str = ' '.join(str.split()

// replace empty space with dash
str = str.replace(" ","-")

// take out any char that matches regex
str = re.sub('[!@#$%^&*()_+<>]', '', str)



2016-12-22 19:55
by perfecto25


These days I am diving into scheme, and now I think am good at recursing and eval. HAHAHA. Just share some new ways:

first ,eval it

print eval('string%s' % (''.join(['.replace("%s","")'%i for i in replace_list])))

second , recurse it

def repn(string,replace_list):
    if replace_list==[]:
        return string
        return repn(string.replace(replace_list.pop(),""),replace_list)

print repn(string,replace_list)

Hey ,don't downvote. I am just want to share some new idea.

2013-10-28 14:23
by tcpiper


How about this - a one liner.

reduce(lambda x,y : x.replace(y,"") ,[',', '!', '.', ';'],";Test , ,  !Stri!ng ..")
2015-11-03 04:13
by Akshay Hazari


i think this is simple enough and will do!

list = [",",",","!",";",":"] #the list goes on.....

theString = "dlkaj;lkdjf'adklfaj;lsd'fa'dfj;alkdjf" #is an example string;
newString="" #the unwanted character free string
for i in range(len(TheString)):
    if theString[i] in list:
        newString += "" #concatenate an empty string.
        newString += theString[i]

this is one way to do it. But if you are tired of keeping a list of characters that you want to remove, you can actually do it by using the order number of the strings you iterate through. the order number is the ascii value of that character. the ascii number for 0 as a char is 48 and the ascii number for lower case z is 122 so:

theString = "lkdsjf;alkd8a'asdjf;lkaheoialkdjf;ad"
newString = ""
for i in range(len(theString)):
     if ord(theString[i]) < 48 or ord(theString[i]) > 122: #ord() => ascii num.
         newString += ""
        newString += theString[i]
2016-11-13 15:45
by Hiskel Kelemework


I am thinking about a solution for this. First I would make the string input as a list. Then I would replace the items of list. Then through using join command, I will return list as a string. The code can be like this:

def the_replacer(text):
    test = []    
    for m in range(len(text)):
        if test[m]==','\
        or test[m]=='!'\
        or test[m]=='.'\
        or test[m]=='\''\
        or test[m]==';':
    return ''.join(test)

This would remove anything from the string. What do you think about that?

2015-05-26 04:00
by Sheikh Ahmad Shah


Here is a more_itertools approach:

import more_itertools as mit

s = "A.B!C?D_E@F#"
blacklist = ".!?_@#"

"".join(mit.flatten(mit.split_at(s, pred=lambda x: x in set(blacklist))))

Here we split upon items found in the blacklist, flatten the results and join the string.

2018-02-09 01:31
by pylang